-- Logs begin at Wed 2022-07-06 12:57:07 UTC, end at Wed 2022-07-06 14:07:59 UTC. -- Jul 06 13:31:12.178839 dv-cinder-819149-8-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 06 13:31:12.187972 dv-cinder-819149-8-iscsi devstack@keystone.service[86840]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 06 13:31:12.189927 dv-cinder-819149-8-iscsi devstack@keystone.service[86840]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jul 6 13:31:12 2022] *** Jul 06 13:31:12.189927 dv-cinder-819149-8-iscsi devstack@keystone.service[86840]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 06 13:31:12.189927 dv-cinder-819149-8-iscsi devstack@keystone.service[86840]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 06 13:31:12.189927 dv-cinder-819149-8-iscsi devstack@keystone.service[86840]: nodename: dv-cinder-819149-8-iscsi Jul 06 13:31:12.189927 dv-cinder-819149-8-iscsi devstack@keystone.service[86840]: machine: x86_64 Jul 06 13:31:12.189927 dv-cinder-819149-8-iscsi devstack@keystone.service[86840]: clock source: unix Jul 06 13:31:12.189927 dv-cinder-819149-8-iscsi devstack@keystone.service[86840]: pcre jit disabled Jul 06 13:31:12.189927 dv-cinder-819149-8-iscsi devstack@keystone.service[86840]: detected number of CPU cores: 4 Jul 06 13:31:12.189927 dv-cinder-819149-8-iscsi devstack@keystone.service[86840]: current working directory: / Jul 06 13:31:12.189927 dv-cinder-819149-8-iscsi devstack@keystone.service[86840]: detected binary path: /usr/bin/uwsgi-core Jul 06 13:31:12.190448 dv-cinder-819149-8-iscsi devstack@keystone.service[86840]: your processes number limit is 47680 Jul 06 13:31:12.190448 dv-cinder-819149-8-iscsi devstack@keystone.service[86840]: your memory page size is 4096 bytes Jul 06 13:31:12.190448 dv-cinder-819149-8-iscsi devstack@keystone.service[86840]: detected max file descriptor number: 2048 Jul 06 13:31:12.190448 dv-cinder-819149-8-iscsi devstack@keystone.service[86840]: lock engine: pthread robust mutexes Jul 06 13:31:12.190448 dv-cinder-819149-8-iscsi devstack@keystone.service[86840]: thunder lock: enabled Jul 06 13:31:12.190448 dv-cinder-819149-8-iscsi devstack@keystone.service[86840]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 06 13:31:12.190448 dv-cinder-819149-8-iscsi devstack@keystone.service[86840]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 06 13:31:12.234124 dv-cinder-819149-8-iscsi devstack@keystone.service[86840]: Python main interpreter initialized at 0x55bce4e479c0 Jul 06 13:31:12.234124 dv-cinder-819149-8-iscsi devstack@keystone.service[86840]: python threads support enabled Jul 06 13:31:12.234124 dv-cinder-819149-8-iscsi devstack@keystone.service[86840]: your server socket listen backlog is limited to 100 connections Jul 06 13:31:12.234124 dv-cinder-819149-8-iscsi devstack@keystone.service[86840]: your mercy for graceful operations on workers is 90 seconds Jul 06 13:31:12.234465 dv-cinder-819149-8-iscsi devstack@keystone.service[86840]: mapped 403077 bytes (393 KB) for 2 cores Jul 06 13:31:12.234540 dv-cinder-819149-8-iscsi devstack@keystone.service[86840]: *** Operational MODE: preforking *** Jul 06 13:31:12.234804 dv-cinder-819149-8-iscsi devstack@keystone.service[86840]: *** uWSGI is running in multiple interpreter mode *** Jul 06 13:31:12.234804 dv-cinder-819149-8-iscsi devstack@keystone.service[86840]: spawned uWSGI master process (pid: 86840) Jul 06 13:31:12.235001 dv-cinder-819149-8-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 06 13:31:12.241630 dv-cinder-819149-8-iscsi devstack@keystone.service[86840]: spawned uWSGI worker 1 (pid: 86849, cores: 1) Jul 06 13:31:12.241630 dv-cinder-819149-8-iscsi devstack@keystone.service[86840]: spawned uWSGI worker 2 (pid: 86850, cores: 1) Jul 06 13:31:12.241630 dv-cinder-819149-8-iscsi devstack@keystone.service[86840]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 06 13:31:17.692209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 06 13:31:17.692209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: from cryptography.utils import int_from_bytes Jul 06 13:31:17.692968 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 06 13:31:17.692968 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: from cryptography.utils import int_from_bytes Jul 06 13:31:17.693124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 06 13:31:17.693124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: from cryptography.utils import int_from_bytes Jul 06 13:31:17.693124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 06 13:31:17.693124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: from cryptography.utils import int_from_bytes Jul 06 13:31:17.741510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 06 13:31:17.741674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 06 13:31:17.741753 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 06 13:31:17.741843 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 06 13:31:17.741916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] command line args: [] {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 06 13:31:17.741997 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] command line args: [] {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 06 13:31:17.742143 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 06 13:31:17.742229 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 06 13:31:17.742328 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ================================================================================ {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 06 13:31:17.742415 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ================================================================================ {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 06 13:31:17.742581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] admin_token = **** {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.742672 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] admin_token = **** {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.742823 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] config_dir = [] {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.742902 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] config_dir = [] {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.743008 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.743094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.743176 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] config_source = [] {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.743247 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] config_source = [] {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.743353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] debug = True {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.743433 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] debug = True {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.743548 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.743850 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.744001 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.744100 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.744202 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.744303 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.744411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.744513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.744610 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.744712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.744823 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] list_limit = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.744925 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] list_limit = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.745071 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] log_config_append = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.745176 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] log_config_append = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.745278 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.745379 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.745480 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] log_dir = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.745581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] log_dir = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.745694 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] log_file = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.745795 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] log_file = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.745906 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.746010 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.746128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.746247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.746330 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.746446 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.746520 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.746644 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.746724 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.746804 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.746866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.746995 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.747072 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.747188 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.747263 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.747390 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.747507 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.747587 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.747661 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.747795 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.747880 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.748005 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.748086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.748158 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.748272 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.748353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.748430 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.748510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.748676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.749723 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.749863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.749941 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.750074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] publish_errors = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.750157 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] publish_errors = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.750229 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.750353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.750429 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.750543 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.750662 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.750743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.750870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.750967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.751051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.751124 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.751199 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] standard_threads = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.751333 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] standard_threads = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.751455 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.751529 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.751647 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.751733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.751810 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.751893 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.751966 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] use_journal = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.752041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] use_journal = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.752143 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] use_json = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.752291 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] use_json = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.752373 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] use_stderr = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.752458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] use_stderr = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.752536 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] use_syslog = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.752613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] use_syslog = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.752689 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.752764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 13:31:17.752932 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.753074 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.753199 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.753276 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.753454 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.753537 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.753691 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.753778 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.753936 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.754012 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.754174 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.754252 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.754420 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.754508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.754652 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.754729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.754896 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.754973 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.755140 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.755228 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.755385 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.755467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.755615 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.755692 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.755858 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.755945 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.756091 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.756173 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.756332 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.756451 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.756570 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.756765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.756883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.757022 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.757151 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.757268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.757382 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.757526 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.757649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.757764 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.757877 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.757997 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.758111 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.758232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.758232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.758460 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.758574 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.758686 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.758799 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.758921 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.759036 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.759149 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.759261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.759381 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.759512 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.759630 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.759744 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.759857 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.759970 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.760083 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.760203 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.760323 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.760438 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.760551 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.760663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.760783 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.760907 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.761047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.761164 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.761278 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.761391 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.761505 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.761630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.761746 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.761865 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.761979 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.762092 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.762211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.762399 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.762517 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.762630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.762749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.762863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.762982 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.763103 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.763217 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.763329 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.763442 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.763556 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.763668 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.763789 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.763905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.764017 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.764137 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.764251 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.764364 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.764475 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.765106 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.765106 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.765305 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.765402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.765517 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.765640 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.765755 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.765868 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.765981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.766094 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.766094 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.766349 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.766465 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.766578 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.766693 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.766807 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.766920 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.767042 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.767157 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.767271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.767390 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.767505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.767632 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.767749 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.767749 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.767974 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.768105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.768221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.768334 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.768457 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.768573 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.768701 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.768824 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.768941 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.768941 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.769208 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.769334 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.769450 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.769564 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.769668 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.769770 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.769889 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.770012 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.770133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.770248 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.770369 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.770484 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.770604 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.770726 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.770843 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.770958 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.771072 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.771186 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.771300 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.771421 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.771537 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.771657 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.771772 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.771893 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.772008 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.772130 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.772246 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.772360 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.772474 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.772577 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.772690 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.772812 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.772936 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.773066 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.773183 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.773297 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.773411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.773537 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.773654 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.773774 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.773895 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.774012 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.774132 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.774323 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.774441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.774554 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.774675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.774790 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.774910 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.775034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.775158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.775285 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.775401 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.775523 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.775639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.775754 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.775868 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.775991 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.776108 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.776221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.776336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.776450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.776572 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.776688 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.776808 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.776923 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.777084 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.777212 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.777337 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.777452 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.777567 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.777681 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.777795 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.777910 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.778032 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.778147 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] auth.external = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.778263 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.778377 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.778491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] auth.external = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.778611 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.778735 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.778879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.779010 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.779127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] auth.password = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.779243 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.779358 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] auth.token = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.779472 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] auth.password = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.779587 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.779714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] auth.token = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.779831 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.779967 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.780367 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.780522 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.780661 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.780793 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.780913 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.781047 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.781167 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.781283 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.781402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.781531 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.781650 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] credential.caching = True {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.781766 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.781882 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.781997 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] credential.caching = True {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.782111 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.782236 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.782353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.782470 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.782593 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.782593 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.782811 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.782940 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.783056 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.783056 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.783272 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.783388 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.783510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.783635 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.783753 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.783877 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.783995 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.784110 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.784225 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.784349 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.784467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.784583 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.784704 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.784876 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.785022 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.785155 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.785274 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.785390 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.785506 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] federation.caching = True {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.785506 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.785722 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.785722 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] federation.caching = True {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.785949 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.786064 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.786178 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.786178 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.786392 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.786524 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.786643 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.786643 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.786858 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.786973 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.787088 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.787211 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.787328 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.787443 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.787559 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.787675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.787798 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.787914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.788030 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.788145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.788259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] identity.caching = True {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.788374 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.788562 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.788689 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] identity.caching = True {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.788806 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.788921 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.789050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.789050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.789281 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.789281 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.789500 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.789615 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.789731 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.789731 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.789954 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.789954 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.790170 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.790285 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.790404 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.790519 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.790646 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.790764 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.790879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.791000 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.791116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.791231 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.791354 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.791470 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.791585 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.791700 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.791815 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.791930 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.792053 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.792170 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.792290 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.792412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.792528 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.792643 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.792766 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.792884 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.793044 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.793164 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.793278 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.793403 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.793528 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.793645 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.793760 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.793875 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.793992 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.794106 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.794229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.794346 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.794460 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.794575 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.794689 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.794805 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.794931 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.795049 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.795164 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.795279 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.795394 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.795508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.796177 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.796375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.796528 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.796645 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.796761 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.796876 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.797020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.797143 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.797259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.797385 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.797503 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.797618 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.797743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.797859 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.797974 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.798089 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.798204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.798320 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.798444 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.798560 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.798675 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.798790 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.799170 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.799292 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.799416 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.799533 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.799648 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.799763 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.799878 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.799992 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.800116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.800240 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.800359 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.800475 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.800589 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.800712 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.800829 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.800944 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.801117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.801234 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.801349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.user = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.801477 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.801595 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.801710 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.user = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.801825 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.801940 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.802055 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.802178 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.802297 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.802418 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.802535 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.802650 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.802765 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.802888 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.803005 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.803119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.803234 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.803350 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.803465 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.803589 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.803706 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.803821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.803936 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.804051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.804166 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.804288 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.804406 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.804521 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.804642 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.804758 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.804873 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.805013 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.805133 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.805248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.805362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.805478 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.805593 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.805719 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.805838 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.805953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.806067 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.806182 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.806297 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.806420 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.806538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.806653 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.806768 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.806883 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.806997 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.807120 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.807236 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.807351 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.807466 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.807582 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.807698 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.807821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.807937 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.808052 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.808166 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.808281 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.808396 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.808519 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.808642 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.808758 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.808873 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.808988 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.809123 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.809251 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.809368 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.809483 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.809598 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.809714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.809829 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.809956 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.810073 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] resource.caching = True {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.810188 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.810307 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.810422 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] resource.caching = True {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.810536 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.810660 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.810778 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.810893 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.811007 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.811121 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.811286 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.811935 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.812054 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.812170 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.812286 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.812412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.812526 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.812717 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.812838 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.812953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.813110 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] role.caching = True {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.813229 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.813343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] role.driver = sql {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.813467 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] role.caching = True {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.813584 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.813699 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] role.driver = sql {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.813814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.813930 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.814045 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.814169 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.814286 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.814401 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.814550 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.814667 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.814791 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.814791 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.815011 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.815127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.815241 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.815357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.815480 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.815597 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.815712 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.815827 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.815944 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.816059 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.816183 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.816299 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.816415 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.816530 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.816646 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.816646 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.816870 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.816988 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.817124 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.817124 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.817340 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.817455 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.817583 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.817701 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.817816 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.817931 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.818047 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.818162 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.818285 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.818402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.818517 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.818632 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.818747 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.818862 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.818985 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.819111 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.819227 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.819343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.819465 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.819590 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.819707 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.819822 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.819937 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.820052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.820168 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.820291 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.820411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.820526 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.820641 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.820756 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.820870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.820993 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.821156 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.821273 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] token.caching = True {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.821388 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.821503 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.821618 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.821745 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.821863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] token.caching = True {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.821978 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.822093 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.822208 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.822324 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.822446 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.822562 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.822677 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.822792 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.822907 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.823022 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.823145 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.823262 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.823377 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.823491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.823607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.823722 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.823844 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.823960 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.824075 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.824189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.824305 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.824420 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.824542 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.824658 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.824773 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.824888 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.825020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.825138 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.825138 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.825370 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.825486 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.825600 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] database.connection = **** {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.825724 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.825849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.825849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.826074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] database.connection = **** {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.826199 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.826326 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.826444 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.826560 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.826675 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.826791 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.827447 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.827611 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.827851 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.827978 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.828095 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.828210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.828325 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.828451 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.828570 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.828686 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.828802 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.828917 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.829146 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.829284 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.829404 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.829520 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.829636 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.829752 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.829867 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.829991 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.830109 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.830224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.830340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.830460 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.830576 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.830701 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.830825 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86850) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 06 13:31:17.830942 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.830942 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 13:31:17.830942 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 06 13:31:17.858941 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86850) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:17.859101 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86849) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:17.859183 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86850) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:17.859258 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86849) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:17.859379 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86850) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:17.859465 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86849) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:17.867674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86849) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:17.867761 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86850) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:17.896062 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86850) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:17.896285 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86850) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:17.896606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86849) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:17.896834 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86849) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.039815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86849) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.040027 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86850) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.088899 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86849) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.089154 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86849) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.090339 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86850) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.090571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86850) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.153611 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86849) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.153843 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86849) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.153971 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86850) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.154079 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86849) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.154205 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86850) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.154290 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86849) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.154427 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86850) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.154511 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86849) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.154644 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86850) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.154738 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86849) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.154860 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86850) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.155109 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86850) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.179377 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86849) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.179521 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86850) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.179617 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86849) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.179769 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86850) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.179873 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86849) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.180010 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86850) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.180095 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86849) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.180235 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86850) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.180320 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86849) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.180449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86850) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.180534 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86849) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.180661 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86850) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.180744 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86849) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.180870 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86850) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.192257 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86849) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.192557 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86850) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.205576 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86849) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.205805 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86849) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.205919 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86850) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.206030 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86849) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.206114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86850) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.206253 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86849) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.206334 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86850) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.206472 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86850) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.206553 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86849) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.206685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86850) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.206767 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86849) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.206895 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86849) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.206982 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86850) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.207102 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86850) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 13:31:18.224666 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.224888 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 06 13:31:18.225149 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.225359 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 06 13:31:18.225457 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.225667 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 06 13:31:18.225747 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.225871 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 06 13:31:18.225959 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.226139 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.226221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.226399 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.226480 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.226653 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.226732 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.226905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.238061 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.238336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.238417 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.238601 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.238695 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.238879 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.238961 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.239153 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.247113 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86849) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.247375 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86850) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.250547 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86849) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.250792 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86850) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.250873 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.251068 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.251149 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.251346 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.251427 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.251634 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.251755 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.251907 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.252013 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.252160 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.252252 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.252421 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.252503 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.252684 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.252776 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.252945 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.253065 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.253234 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.253316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.253497 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.279647 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.279776 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.281935 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86849) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.282035 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86850) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.282210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.282295 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.287439 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86850) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.287532 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86849) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.287705 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.287789 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.287997 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.288080 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.295362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86850) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.295571 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86849) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.296323 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.296572 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.301521 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86849) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.301695 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86850) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.301833 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.301982 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.302111 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.302239 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.302364 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.302501 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.302630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.302766 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.302894 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.303022 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.303150 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.303277 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.303412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.303542 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.322090 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.322364 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.322510 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.322647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.326956 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86849) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.327283 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86850) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.330610 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.330898 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.331037 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.331191 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.336043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86849) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.336290 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86850) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.339687 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86849) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.339911 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86850) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.343873 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.344166 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.344252 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.344458 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.344539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.344752 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.344834 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.345051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.345134 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.345339 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.345433 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.345616 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.364548 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86849) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.364829 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.364910 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86850) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.365119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.365206 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.365385 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.365466 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.365645 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.373345 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.373548 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.375784 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.375884 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.376059 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.376149 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.379283 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86849) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.379369 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86850) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.379566 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.379647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.379847 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.379940 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.380122 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.380204 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.380392 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.380473 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.394145 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86850) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.394419 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.394514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86849) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.394694 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.394776 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.394972 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.395057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.395247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.395328 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.395523 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.395810 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.396082 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.396863 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.397175 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.415046 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86850) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.415352 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86849) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.418365 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86850) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.418674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86849) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.422442 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.422540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.424365 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.424455 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.426078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86850) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.426163 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86849) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.426355 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.426437 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.426641 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.426724 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.433905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.434038 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.435504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86850) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.435645 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86849) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.439480 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.439571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.439762 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.439844 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.440042 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.440149 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.440317 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.440424 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.449527 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86850) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.449616 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86849) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.449808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.449891 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.450084 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.450166 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.457909 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86850) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.458096 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86849) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 13:31:18.458180 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.458372 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.458462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.458655 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.458747 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.458936 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.459018 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.459204 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.459335 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.459489 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.459610 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.459761 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.459880 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.460041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.460169 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.460403 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.460484 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.460668 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.460748 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.460934 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.461044 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.461221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.461303 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.461482 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86850) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.461561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.461747 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86849) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 13:31:18.499964 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55bce4e479c0 pid: 86850 (default app) Jul 06 13:31:18.501111 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55bce4e479c0 pid: 86849 (default app) Jul 06 13:31:18.503310 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60971dc-a541-4278-b6b4-38247e7137c4 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:18.503601 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60971dc-a541-4278-b6b4-38247e7137c4 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:18.503883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60971dc-a541-4278-b6b4-38247e7137c4 None None] PATH_INFO: `/v3` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:18.508031 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 791 bytes} [Wed Jul 6 13:31:18 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:26.148342 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec342c-bc8d-42c2-8e6d-c85a74420d95 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:26.148891 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec342c-bc8d-42c2-8e6d-c85a74420d95 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:26.148891 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec342c-bc8d-42c2-8e6d-c85a74420d95 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:26.153308 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1/2] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:31:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:31:26.159050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa5f65-3c4a-4f01-bc5c-38a257f2245f None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:26.159439 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa5f65-3c4a-4f01-bc5c-38a257f2245f None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:26.159604 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa5f65-3c4a-4f01-bc5c-38a257f2245f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:26.228046 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: INFO dbcounter [None req-a4aa5f65-3c4a-4f01-bc5c-38a257f2245f None None] Registered counter for database keystone Jul 06 13:31:26.250114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG oslo_db.sqlalchemy.engines [None req-a4aa5f65-3c4a-4f01-bc5c-38a257f2245f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86850) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 06 13:31:26.256221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writer thread running {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 06 13:31:26.434781 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG passlib.handlers.bcrypt [None req-a4aa5f65-3c4a-4f01-bc5c-38a257f2245f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86850) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 06 13:31:26.435292 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG passlib.handlers.bcrypt [None req-a4aa5f65-3c4a-4f01-bc5c-38a257f2245f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86850) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 06 13:31:26.490363 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-a4aa5f65-3c4a-4f01-bc5c-38a257f2245f None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:31:26.491978 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a4aa5f65-3c4a-4f01-bc5c-38a257f2245f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:31:26.613215 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 2/3] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:31:26 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 06 13:31:26.804276 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec342c-bc8d-42c2-8e6d-c85a74420d95 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:26.875781 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: INFO dbcounter [None req-c1ec342c-bc8d-42c2-8e6d-c85a74420d95 None None] Registered counter for database keystone Jul 06 13:31:26.895798 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG oslo_db.sqlalchemy.engines [None req-c1ec342c-bc8d-42c2-8e6d-c85a74420d95 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86849) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 06 13:31:26.901941 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writer thread running {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 06 13:31:27.022339 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec342c-bc8d-42c2-8e6d-c85a74420d95 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:27.026549 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5614f-8823-43fd-a9cc-1bd90d7eec55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:27.027210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5614f-8823-43fd-a9cc-1bd90d7eec55 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:27.027497 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5614f-8823-43fd-a9cc-1bd90d7eec55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:27.027776 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5614f-8823-43fd-a9cc-1bd90d7eec55 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:27.036293 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c5614f-8823-43fd-a9cc-1bd90d7eec55 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:27.090509 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a4c5614f-8823-43fd-a9cc-1bd90d7eec55 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 06 13:31:27.090800 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c5614f-8823-43fd-a9cc-1bd90d7eec55 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:27.100989 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 2/4] 10.222.0.20 () {58 vars in 1324 bytes} [Wed Jul 6 13:31:26 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 06 13:31:27.106989 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa5f65-3c4a-4f01-bc5c-38a257f2245f None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:27.121838 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa5f65-3c4a-4f01-bc5c-38a257f2245f None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:27.126898 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d0c6d-735f-4fa8-aedb-8b2df5e88db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:27.128013 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d0c6d-735f-4fa8-aedb-8b2df5e88db4 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:27.128295 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d0c6d-735f-4fa8-aedb-8b2df5e88db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:27.128570 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d0c6d-735f-4fa8-aedb-8b2df5e88db4 admin admin] PATH_INFO: `/v3/projects/775434dcf39c43beaa4ed6a50089f763` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:27.130094 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38d0c6d-735f-4fa8-aedb-8b2df5e88db4 admin admin] RBAC: Authorizing `identity:get_project(project_id=775434dcf39c43beaa4ed6a50089f763)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:27.182889 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b38d0c6d-735f-4fa8-aedb-8b2df5e88db4 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 06 13:31:27.183431 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38d0c6d-735f-4fa8-aedb-8b2df5e88db4 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:27.188498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 3/5] 10.222.0.20 () {56 vars in 1207 bytes} [Wed Jul 6 13:31:27 2022] GET /identity/v3/projects/775434dcf39c43beaa4ed6a50089f763 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:29.206652 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939e33a-3bf9-481c-abcd-4802795eb6ca None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:29.207185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939e33a-3bf9-481c-abcd-4802795eb6ca None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:29.207185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939e33a-3bf9-481c-abcd-4802795eb6ca None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:29.211289 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 3/6] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:31:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:31:29.216610 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93895130-4e3d-4a94-87e3-84b28ad13139 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:29.217038 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93895130-4e3d-4a94-87e3-84b28ad13139 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:29.217791 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93895130-4e3d-4a94-87e3-84b28ad13139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:29.236328 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-93895130-4e3d-4a94-87e3-84b28ad13139 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:31:29.237877 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-93895130-4e3d-4a94-87e3-84b28ad13139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:31:29.245671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 4/7] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:31:29 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 06 13:31:29.267505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939e33a-3bf9-481c-abcd-4802795eb6ca None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:29.279216 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939e33a-3bf9-481c-abcd-4802795eb6ca None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:29.284069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf7423-5613-4bf6-93d1-1093671a2560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:29.284692 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf7423-5613-4bf6-93d1-1093671a2560 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:29.284976 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf7423-5613-4bf6-93d1-1093671a2560 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:29.285283 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf7423-5613-4bf6-93d1-1093671a2560 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:29.292959 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbf7423-5613-4bf6-93d1-1093671a2560 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:29.294773 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbf7423-5613-4bf6-93d1-1093671a2560 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:29.303963 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 4/8] 10.222.0.20 () {58 vars in 1324 bytes} [Wed Jul 6 13:31:29 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 13:31:29.309338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93895130-4e3d-4a94-87e3-84b28ad13139 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:29.319093 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93895130-4e3d-4a94-87e3-84b28ad13139 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:29.323462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa5c0f-c705-4546-b2ba-bfa30d95e84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:29.324324 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa5c0f-c705-4546-b2ba-bfa30d95e84f admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:29.324611 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa5c0f-c705-4546-b2ba-bfa30d95e84f admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:29.324888 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa5c0f-c705-4546-b2ba-bfa30d95e84f admin admin] PATH_INFO: `/v3/users/c17e13a3d58f4ceab768f46d43e2cab5` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:29.326320 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefa5c0f-c705-4546-b2ba-bfa30d95e84f admin admin] RBAC: Authorizing `identity:get_user(user_id=c17e13a3d58f4ceab768f46d43e2cab5)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:29.328345 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefa5c0f-c705-4546-b2ba-bfa30d95e84f admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:29.329945 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 5/9] 10.222.0.20 () {56 vars in 1198 bytes} [Wed Jul 6 13:31:29 2022] GET /identity/v3/users/c17e13a3d58f4ceab768f46d43e2cab5 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:31.403528 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c5e30-7139-48a2-ba7a-500d6d20163e None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:31.404026 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c5e30-7139-48a2-ba7a-500d6d20163e None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:31.404214 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c5e30-7139-48a2-ba7a-500d6d20163e None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:31.405576 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 5/10] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:31:31.410707 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57028215-3ba1-4b45-8da9-2b8d8e308d29 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:31.411100 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57028215-3ba1-4b45-8da9-2b8d8e308d29 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:31.411270 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57028215-3ba1-4b45-8da9-2b8d8e308d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:31.428656 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-57028215-3ba1-4b45-8da9-2b8d8e308d29 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:31:31.429840 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-57028215-3ba1-4b45-8da9-2b8d8e308d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:31:31.498589 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95278ab6-d4a2-4c3f-9998-34a55741ae14 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:31.498890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95278ab6-d4a2-4c3f-9998-34a55741ae14 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:31.499203 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95278ab6-d4a2-4c3f-9998-34a55741ae14 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:31.500590 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 6/11] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:31:31.505900 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f7b0f-4edd-4732-8437-9d16a992e545 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:31.506190 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f7b0f-4edd-4732-8437-9d16a992e545 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:31.506474 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f7b0f-4edd-4732-8437-9d16a992e545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:31.511411 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 6/12] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:31:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:31:31.535882 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57028215-3ba1-4b45-8da9-2b8d8e308d29 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:31.547072 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG passlib.handlers.bcrypt [None req-367f7b0f-4edd-4732-8437-9d16a992e545 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86849) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 06 13:31:31.547439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG passlib.handlers.bcrypt [None req-367f7b0f-4edd-4732-8437-9d16a992e545 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86849) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 06 13:31:31.559297 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:31.594428 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-367f7b0f-4edd-4732-8437-9d16a992e545 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:31:31.596033 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-367f7b0f-4edd-4732-8437-9d16a992e545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:31:31.639858 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57028215-3ba1-4b45-8da9-2b8d8e308d29 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:31.673174 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb62573-6dce-4b27-ac12-7c8cdc123fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:31.674232 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb62573-6dce-4b27-ac12-7c8cdc123fb3 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:31.674521 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb62573-6dce-4b27-ac12-7c8cdc123fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:31.674797 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb62573-6dce-4b27-ac12-7c8cdc123fb3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:31.687167 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb62573-6dce-4b27-ac12-7c8cdc123fb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:31:31.699060 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 7/13] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:31:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 13:31:31.699941 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-bfb62573-6dce-4b27-ac12-7c8cdc123fb3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:31:31.699941 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:31:31.699941 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:31:31.699941 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:31:31.699941 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:31:31.699941 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:31:31.699941 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:31:31.699941 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:31:31.699941 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:31:31.699941 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:31:31.699941 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:31:31.699941 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:31:31.699941 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:31:31.699941 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:31:31.699941 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:31:31.699941 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:31:31.701034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:31:31.701034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:31:31.701034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:31:31.701034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:31:31.701034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:31:31.701034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:31:31.701034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:31:31.701034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:31:31.701034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:31:31.701034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:31:31.701034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:31:31.701034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:31:31.701034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:31:31.701034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:31:31.701034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:31:31.701034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:31:31.703381 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:31:31.703381 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:31:31.703381 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:31:31.703381 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:31:31.703381 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:31:31.703381 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:31:31.703381 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:31:31.703381 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:31:31.703381 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:31:31.703381 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:31:31.703381 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:31:31.703381 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:31:31.703381 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 7/14] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:31:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:31:31.707531 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f7b0f-4edd-4732-8437-9d16a992e545 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:31.720905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb62573-6dce-4b27-ac12-7c8cdc123fb3 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:31.803399 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:31.816724 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f7b0f-4edd-4732-8437-9d16a992e545 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:31.819048 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:31.822959 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb62573-6dce-4b27-ac12-7c8cdc123fb3 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:31.855590 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a5d3c8-9d5b-4e01-ba2b-16a821fd952b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:31.856531 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5d3c8-9d5b-4e01-ba2b-16a821fd952b None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:31.856684 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b9c2c-3e9c-4d49-828e-1f592cb3ea46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:31.856822 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5d3c8-9d5b-4e01-ba2b-16a821fd952b None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:31.857142 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5d3c8-9d5b-4e01-ba2b-16a821fd952b None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:31.857362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b9c2c-3e9c-4d49-828e-1f592cb3ea46 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:31.857985 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b9c2c-3e9c-4d49-828e-1f592cb3ea46 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:31.858097 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a5d3c8-9d5b-4e01-ba2b-16a821fd952b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:31:31.858272 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b9c2c-3e9c-4d49-828e-1f592cb3ea46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:31.858412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a5d3c8-9d5b-4e01-ba2b-16a821fd952b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:31.859862 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a5d3c8-9d5b-4e01-ba2b-16a821fd952b None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:31.871447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 8/15] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:31:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:31.877655 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a5d3c8-9d5b-4e01-ba2b-16a821fd952b None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:31.927066 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b9c2c-3e9c-4d49-828e-1f592cb3ea46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:31.928531 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b9c2c-3e9c-4d49-828e-1f592cb3ea46 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:31.976417 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a5d3c8-9d5b-4e01-ba2b-16a821fd952b None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:32.009068 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:32.009691 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea2c1e0-305e-4613-9efa-1961a019d4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:32.010332 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2c1e0-305e-4613-9efa-1961a019d4d7 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:32.010592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2c1e0-305e-4613-9efa-1961a019d4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:32.010875 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2c1e0-305e-4613-9efa-1961a019d4d7 None admin] PATH_INFO: `/v3/users/c17e13a3d58f4ceab768f46d43e2cab5` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:32.012510 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea2c1e0-305e-4613-9efa-1961a019d4d7 None admin] RBAC: Authorizing `identity:get_user(user_id=c17e13a3d58f4ceab768f46d43e2cab5)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:32.014589 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea2c1e0-305e-4613-9efa-1961a019d4d7 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:32.016462 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 9/16] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:31:31 2022] GET /identity/v3/users/c17e13a3d58f4ceab768f46d43e2cab5 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:31:32.021491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea2c1e0-305e-4613-9efa-1961a019d4d7 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:32.028567 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 8/17] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:31:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 06 13:31:32.034606 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b9c2c-3e9c-4d49-828e-1f592cb3ea46 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:32.055856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:32.123840 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea2c1e0-305e-4613-9efa-1961a019d4d7 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:32.130933 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b9c2c-3e9c-4d49-828e-1f592cb3ea46 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:32.156625 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ead45-6eca-49e8-9cfc-e2992cec0c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:32.157444 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ead45-6eca-49e8-9cfc-e2992cec0c16 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:32.157727 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ead45-6eca-49e8-9cfc-e2992cec0c16 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:32.158004 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ead45-6eca-49e8-9cfc-e2992cec0c16 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:32.159309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ead45-6eca-49e8-9cfc-e2992cec0c16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:32.160754 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ead45-6eca-49e8-9cfc-e2992cec0c16 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:32.162557 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 10/18] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:31:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:31:32.163839 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55015860-09cf-4cdd-b1dd-06a9775420d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:32.164641 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55015860-09cf-4cdd-b1dd-06a9775420d2 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:32.164926 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55015860-09cf-4cdd-b1dd-06a9775420d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:32.165244 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55015860-09cf-4cdd-b1dd-06a9775420d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:32.167996 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ead45-6eca-49e8-9cfc-e2992cec0c16 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:32.176260 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55015860-09cf-4cdd-b1dd-06a9775420d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:32.177761 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55015860-09cf-4cdd-b1dd-06a9775420d2 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:32.195437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-55015860-09cf-4cdd-b1dd-06a9775420d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:31:32.195437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:31:32.195437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:31:32.195437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:31:32.195437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:31:32.195437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:31:32.195437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:31:32.195437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:31:32.195437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:31:32.195437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:31:32.195437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:31:32.195437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:31:32.195437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:31:32.195437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:31:32.195437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:31:32.196237 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:31:32.196237 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:31:32.196237 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:31:32.196237 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:31:32.196237 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:31:32.196237 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:31:32.196237 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:31:32.196237 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:31:32.196237 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:31:32.196237 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:31:32.196237 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:31:32.196237 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:31:32.196237 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:31:32.196237 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:31:32.196237 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:31:32.196237 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:31:32.196237 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:31:32.197118 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:31:32.197118 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:31:32.197118 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:31:32.197118 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:31:32.197118 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 9/19] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:31:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:31:32.203058 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55015860-09cf-4cdd-b1dd-06a9775420d2 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:32.236866 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:32.267590 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ead45-6eca-49e8-9cfc-e2992cec0c16 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:32.269115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:32.301962 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e56f41a-f33a-4f1f-9d1e-4c2d5f49706d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:32.302631 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56f41a-f33a-4f1f-9d1e-4c2d5f49706d None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:32.302918 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56f41a-f33a-4f1f-9d1e-4c2d5f49706d None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:32.303199 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56f41a-f33a-4f1f-9d1e-4c2d5f49706d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:32.304046 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e56f41a-f33a-4f1f-9d1e-4c2d5f49706d None admin] RBAC: Adding query filter params (role.id=795daa2c78354fd48e4a741e1182cc98, scope.domain.id=default, user.id=c17e13a3d58f4ceab768f46d43e2cab5) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:31:32.304361 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e56f41a-f33a-4f1f-9d1e-4c2d5f49706d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:32.305678 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55015860-09cf-4cdd-b1dd-06a9775420d2 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:32.305917 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e56f41a-f33a-4f1f-9d1e-4c2d5f49706d None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:32.315545 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 11/20] 10.222.0.20 () {56 vars in 1322 bytes} [Wed Jul 6 13:31:32 2022] GET /identity/v3/role_assignments?user.id=c17e13a3d58f4ceab768f46d43e2cab5&scope.domain.id=default&role.id=795daa2c78354fd48e4a741e1182cc98 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:31:32.341034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87d9f1-5051-4582-8431-a131fa842d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:32.341706 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87d9f1-5051-4582-8431-a131fa842d08 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:32.342568 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87d9f1-5051-4582-8431-a131fa842d08 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:32.342848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87d9f1-5051-4582-8431-a131fa842d08 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:32.343713 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e87d9f1-5051-4582-8431-a131fa842d08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:31:32.344036 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e87d9f1-5051-4582-8431-a131fa842d08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:32.345500 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e87d9f1-5051-4582-8431-a131fa842d08 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:32.365327 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 10/21] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:31:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:31:34.687546 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e43ab-f1ad-4fce-b6b9-2ad8b63c41d0 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:34.688542 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e43ab-f1ad-4fce-b6b9-2ad8b63c41d0 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:34.689139 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e43ab-f1ad-4fce-b6b9-2ad8b63c41d0 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:34.690710 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 12/22] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:31:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:31:34.696168 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d86089-b034-44fd-986c-8f5adfa5f731 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:34.696987 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d86089-b034-44fd-986c-8f5adfa5f731 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:34.697478 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d86089-b034-44fd-986c-8f5adfa5f731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:34.722073 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-d5d86089-b034-44fd-986c-8f5adfa5f731 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:31:34.723589 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-d5d86089-b034-44fd-986c-8f5adfa5f731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:31:34.811308 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 11/23] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:31:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:31:34.846307 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e43ab-f1ad-4fce-b6b9-2ad8b63c41d0 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:34.979549 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e43ab-f1ad-4fce-b6b9-2ad8b63c41d0 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:35.020416 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7afaf9-7e7d-493f-bdf9-24318ed1ac0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:35.021609 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7afaf9-7e7d-493f-bdf9-24318ed1ac0e None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:35.022155 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7afaf9-7e7d-493f-bdf9-24318ed1ac0e None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:35.022625 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7afaf9-7e7d-493f-bdf9-24318ed1ac0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:35.038379 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7afaf9-7e7d-493f-bdf9-24318ed1ac0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:31:35.053703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-6e7afaf9-7e7d-493f-bdf9-24318ed1ac0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:31:35.053703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:31:35.053703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:31:35.053703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:31:35.053703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:31:35.053703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:31:35.053703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:31:35.053703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:31:35.053703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:31:35.053703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:31:35.053703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:31:35.053703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:31:35.053703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:31:35.053703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:31:35.053703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:31:35.053703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:31:35.054552 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:31:35.054552 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:31:35.054552 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:31:35.054552 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:31:35.054552 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:31:35.054552 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:31:35.054552 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:31:35.054552 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:31:35.054552 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:31:35.054552 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:31:35.054552 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:31:35.054552 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:31:35.054552 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:31:35.054552 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:31:35.054552 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:31:35.054552 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:31:35.055310 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:31:35.055310 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:31:35.055310 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:31:35.055310 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:31:35.055310 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:31:35.055310 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:31:35.055310 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:31:35.055310 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:31:35.055310 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:31:35.055310 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:31:35.055310 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:31:35.055310 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:31:35.057706 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 13/24] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:31:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:31:35.063987 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d86089-b034-44fd-986c-8f5adfa5f731 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:35.093637 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:35.189292 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d86089-b034-44fd-986c-8f5adfa5f731 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:35.234410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32215078-7654-46da-8e0b-a863f7518ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:35.235402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32215078-7654-46da-8e0b-a863f7518ccc None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:35.235889 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32215078-7654-46da-8e0b-a863f7518ccc None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:35.236379 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32215078-7654-46da-8e0b-a863f7518ccc None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:35.237368 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32215078-7654-46da-8e0b-a863f7518ccc None admin] RBAC: Adding query filter params (name=admin) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:31:35.237878 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32215078-7654-46da-8e0b-a863f7518ccc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:35.239469 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32215078-7654-46da-8e0b-a863f7518ccc None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:35.255004 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 12/25] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:31:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:35.261088 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7afaf9-7e7d-493f-bdf9-24318ed1ac0e None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:35.312826 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8058c-ebcf-4597-a64f-9daf8763f9af None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:35.313573 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8058c-ebcf-4597-a64f-9daf8763f9af None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:35.313808 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:35.314737 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8058c-ebcf-4597-a64f-9daf8763f9af None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:35.316579 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 13/26] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:31:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:31:35.321913 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c1113-66fc-44ba-bb02-2bb4ecd7073b None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:35.322411 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c1113-66fc-44ba-bb02-2bb4ecd7073b None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:35.322889 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c1113-66fc-44ba-bb02-2bb4ecd7073b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:35.362012 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-484c1113-66fc-44ba-bb02-2bb4ecd7073b None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:31:35.363537 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-484c1113-66fc-44ba-bb02-2bb4ecd7073b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:31:35.410875 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7afaf9-7e7d-493f-bdf9-24318ed1ac0e None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:35.471084 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735ce00-45cd-4fdd-8987-076d611f4eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:35.471975 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735ce00-45cd-4fdd-8987-076d611f4eec None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:35.472261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735ce00-45cd-4fdd-8987-076d611f4eec None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:35.472544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735ce00-45cd-4fdd-8987-076d611f4eec None admin] PATH_INFO: `/v3/users/c17e13a3d58f4ceab768f46d43e2cab5` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:35.477774 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b735ce00-45cd-4fdd-8987-076d611f4eec None admin] RBAC: Authorizing `identity:get_user(user_id=c17e13a3d58f4ceab768f46d43e2cab5)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:35.485086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b735ce00-45cd-4fdd-8987-076d611f4eec None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:35.485086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 14/27] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:31:35 2022] GET /identity/v3/users/c17e13a3d58f4ceab768f46d43e2cab5 => generated 247 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:35.488459 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 14/28] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:31:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 13:31:35.491088 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735ce00-45cd-4fdd-8987-076d611f4eec None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:35.524016 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484c1113-66fc-44ba-bb02-2bb4ecd7073b None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:35.558099 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:35.621249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:35.629984 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735ce00-45cd-4fdd-8987-076d611f4eec None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:35.665866 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484c1113-66fc-44ba-bb02-2bb4ecd7073b None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:35.667424 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78314f01-a0ee-4372-86d0-8383fd9984eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:35.668347 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78314f01-a0ee-4372-86d0-8383fd9984eb None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:35.668631 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78314f01-a0ee-4372-86d0-8383fd9984eb None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:35.668908 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78314f01-a0ee-4372-86d0-8383fd9984eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:35.670643 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78314f01-a0ee-4372-86d0-8383fd9984eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:35.672177 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78314f01-a0ee-4372-86d0-8383fd9984eb None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:35.673925 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 15/29] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:31:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:35.675497 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b68fc0-a0f1-4317-a22d-132ddaffd417 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:35.675784 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b68fc0-a0f1-4317-a22d-132ddaffd417 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:35.676061 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b68fc0-a0f1-4317-a22d-132ddaffd417 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:35.677493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 16/30] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:31:35.681785 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7628c-85c4-49f2-a600-8296507b75d2 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:35.681785 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7628c-85c4-49f2-a600-8296507b75d2 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:35.681785 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7628c-85c4-49f2-a600-8296507b75d2 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:35.681785 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 17/31] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:31:35.685249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7628c-85c4-49f2-a600-8296507b75d2 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:35.710507 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a1c3d9-edaa-4cc4-8216-f68a41c7411f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:35.711300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1c3d9-edaa-4cc4-8216-f68a41c7411f None admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:35.711657 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1c3d9-edaa-4cc4-8216-f68a41c7411f None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:35.712092 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1c3d9-edaa-4cc4-8216-f68a41c7411f None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:35.713089 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1c3d9-edaa-4cc4-8216-f68a41c7411f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:35.714562 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1c3d9-edaa-4cc4-8216-f68a41c7411f None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:35.801626 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7628c-85c4-49f2-a600-8296507b75d2 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:35.839342 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c92fd-e72c-4750-98a2-658078350293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:35.840314 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c92fd-e72c-4750-98a2-658078350293 None admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:35.840605 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c92fd-e72c-4750-98a2-658078350293 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:35.840889 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c92fd-e72c-4750-98a2-658078350293 None admin] PATH_INFO: `/v3/domains/default/users/c17e13a3d58f4ceab768f46d43e2cab5/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:35.843156 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c92fd-e72c-4750-98a2-658078350293 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c17e13a3d58f4ceab768f46d43e2cab5, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:35.844620 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c92fd-e72c-4750-98a2-658078350293 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:35.850121 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 15/32] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Jul 6 13:31:35 2022] POST /identity/v3/roles => generated 225 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:31:35.851830 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176e73a-f206-4b85-8b8c-3b4ce75042cd None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:35.852154 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176e73a-f206-4b85-8b8c-3b4ce75042cd None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:35.852462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176e73a-f206-4b85-8b8c-3b4ce75042cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:35.873873 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-3176e73a-f206-4b85-8b8c-3b4ce75042cd None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:31:35.875548 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-3176e73a-f206-4b85-8b8c-3b4ce75042cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:31:35.955178 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 16/33] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:31:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:31:35.956583 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbb996-f170-4f88-8a60-1335d27d94ff None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:35.956927 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbb996-f170-4f88-8a60-1335d27d94ff None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:35.957239 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbb996-f170-4f88-8a60-1335d27d94ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:35.969694 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:35.976615 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-fcfbb996-f170-4f88-8a60-1335d27d94ff None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:31:35.978359 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-fcfbb996-f170-4f88-8a60-1335d27d94ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:31:35.996178 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 18/34] 10.222.0.20 () {58 vars in 1361 bytes} [Wed Jul 6 13:31:35 2022] PUT /identity/v3/domains/default/users/c17e13a3d58f4ceab768f46d43e2cab5/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:31:36.007006 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c92fd-e72c-4750-98a2-658078350293 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:36.070681 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 17/35] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:31:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:31:36.077659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:36.111773 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfbb996-f170-4f88-8a60-1335d27d94ff None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:36.214789 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c92fd-e72c-4750-98a2-658078350293 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:36.258625 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5ece4-87cc-498f-926b-27af94145014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:36.259470 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5ece4-87cc-498f-926b-27af94145014 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:36.259768 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5ece4-87cc-498f-926b-27af94145014 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:36.260055 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5ece4-87cc-498f-926b-27af94145014 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:36.284379 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5ece4-87cc-498f-926b-27af94145014 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:36.286218 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5ece4-87cc-498f-926b-27af94145014 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:36.314114 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-e9d5ece4-87cc-498f-926b-27af94145014 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:31:36.314114 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:31:36.314114 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:31:36.314114 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:31:36.314114 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:31:36.314114 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:31:36.314114 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:31:36.314114 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:31:36.314114 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:31:36.314114 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:31:36.314114 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:31:36.314114 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:31:36.314114 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:31:36.314114 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:31:36.314114 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:31:36.314981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:31:36.314981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:31:36.314981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:31:36.314981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:31:36.314981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:31:36.314981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:31:36.314981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:31:36.314981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:31:36.314981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:31:36.314981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:31:36.314981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:31:36.314981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:31:36.314981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:31:36.314981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:31:36.314981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:31:36.314981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:31:36.314981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:31:36.316237 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:31:36.316237 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:31:36.316237 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:31:36.316237 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:31:36.316237 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 19/36] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:31:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:31:36.316864 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd256760-32d8-420b-b211-001935f55dc8 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:36.317348 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd256760-32d8-420b-b211-001935f55dc8 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:36.317761 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd256760-32d8-420b-b211-001935f55dc8 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:36.322148 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 20/37] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:31:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:31:36.323035 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd256760-32d8-420b-b211-001935f55dc8 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:36.394159 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfbb996-f170-4f88-8a60-1335d27d94ff None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:36.425957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:36.436394 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9ae8e-619f-4aef-b3c6-dea41e8ef548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:36.437495 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9ae8e-619f-4aef-b3c6-dea41e8ef548 None admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:36.437985 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9ae8e-619f-4aef-b3c6-dea41e8ef548 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:36.438468 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9ae8e-619f-4aef-b3c6-dea41e8ef548 None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:36.439539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f9ae8e-619f-4aef-b3c6-dea41e8ef548 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:36.441419 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f9ae8e-619f-4aef-b3c6-dea41e8ef548 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:36.445688 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:36.462036 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd256760-32d8-420b-b211-001935f55dc8 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:36.462891 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 18/38] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Jul 6 13:31:36 2022] POST /identity/v3/roles => generated 219 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:31:36.466137 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46ec9d-f66e-43ce-b9b3-81d854633a55 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:36.466660 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46ec9d-f66e-43ce-b9b3-81d854633a55 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:36.467703 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46ec9d-f66e-43ce-b9b3-81d854633a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:36.506941 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-9d46ec9d-f66e-43ce-b9b3-81d854633a55 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:31:36.510079 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-9d46ec9d-f66e-43ce-b9b3-81d854633a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:31:36.578707 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89679943-cad1-477f-8538-63a05d9c4cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:36.579307 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89679943-cad1-477f-8538-63a05d9c4cc9 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:36.579601 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89679943-cad1-477f-8538-63a05d9c4cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:36.579881 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89679943-cad1-477f-8538-63a05d9c4cc9 None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:36.580650 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89679943-cad1-477f-8538-63a05d9c4cc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:31:36.580962 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89679943-cad1-477f-8538-63a05d9c4cc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:36.582445 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89679943-cad1-477f-8538-63a05d9c4cc9 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:36.596127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 19/39] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:31:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:31:36.611150 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 21/40] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:31:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:36.616800 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46ec9d-f66e-43ce-b9b3-81d854633a55 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:36.620970 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89679943-cad1-477f-8538-63a05d9c4cc9 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:36.738081 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46ec9d-f66e-43ce-b9b3-81d854633a55 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:36.746212 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89679943-cad1-477f-8538-63a05d9c4cc9 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:36.771709 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:36.782246 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656aa5b8-0e5c-40cc-b01b-1ff8b0d1eddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:36.782927 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656aa5b8-0e5c-40cc-b01b-1ff8b0d1eddc None admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:36.783217 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656aa5b8-0e5c-40cc-b01b-1ff8b0d1eddc None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:36.783496 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656aa5b8-0e5c-40cc-b01b-1ff8b0d1eddc None admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:36.784231 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656aa5b8-0e5c-40cc-b01b-1ff8b0d1eddc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:36.785730 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656aa5b8-0e5c-40cc-b01b-1ff8b0d1eddc None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:36.787448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb245a85-4d35-4f39-9d32-e043fe960e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:36.788139 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb245a85-4d35-4f39-9d32-e043fe960e31 None admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:36.788427 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb245a85-4d35-4f39-9d32-e043fe960e31 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:36.788710 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb245a85-4d35-4f39-9d32-e043fe960e31 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:36.789648 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb245a85-4d35-4f39-9d32-e043fe960e31 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:36.791262 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb245a85-4d35-4f39-9d32-e043fe960e31 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:36.794496 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:36.818914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 22/41] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Jul 6 13:31:36 2022] POST /identity/v3/roles => generated 223 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:31:36.818914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec01b7-65d7-4e8d-8f0d-249dc320914d None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:36.818914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec01b7-65d7-4e8d-8f0d-249dc320914d None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:36.818914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec01b7-65d7-4e8d-8f0d-249dc320914d None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:36.818914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 23/42] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:31:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:31:36.820815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece761c-3373-4af9-8ad9-9a6cf979a31f None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:36.820918 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece761c-3373-4af9-8ad9-9a6cf979a31f None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:36.821805 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece761c-3373-4af9-8ad9-9a6cf979a31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:36.830287 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 20/43] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Jul 6 13:31:36 2022] POST /identity/v3/projects => generated 301 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:31:36.840519 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-aece761c-3373-4af9-8ad9-9a6cf979a31f None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:31:36.843133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-aece761c-3373-4af9-8ad9-9a6cf979a31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:31:36.937512 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 24/44] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:31:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 13:31:36.962465 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656aa5b8-0e5c-40cc-b01b-1ff8b0d1eddc None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:37.077883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656aa5b8-0e5c-40cc-b01b-1ff8b0d1eddc None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:37.119156 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb205c-5561-41cb-96f6-c7ef781def0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:37.119156 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb205c-5561-41cb-96f6-c7ef781def0f None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:37.119156 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb205c-5561-41cb-96f6-c7ef781def0f None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:37.119156 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb205c-5561-41cb-96f6-c7ef781def0f None admin] PATH_INFO: `/v3/domains/default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:37.119632 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb205c-5561-41cb-96f6-c7ef781def0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:37.121142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb205c-5561-41cb-96f6-c7ef781def0f None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:37.122767 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 21/45] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:31:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:37.128930 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece761c-3373-4af9-8ad9-9a6cf979a31f None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:37.227280 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece761c-3373-4af9-8ad9-9a6cf979a31f None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:37.250794 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:37.262034 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f4423-0a11-4942-ac9b-6bd251898051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:37.262707 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f4423-0a11-4942-ac9b-6bd251898051 None admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:37.263048 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f4423-0a11-4942-ac9b-6bd251898051 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:37.263334 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f4423-0a11-4942-ac9b-6bd251898051 None admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:37.264444 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66f4423-0a11-4942-ac9b-6bd251898051 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:37.265925 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66f4423-0a11-4942-ac9b-6bd251898051 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:37.294063 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 25/46] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Jul 6 13:31:37 2022] POST /identity/v3/projects => generated 312 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:31:38.261765 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d31d5-68c2-44d8-8a2b-11de0572d247 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:38.262275 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d31d5-68c2-44d8-8a2b-11de0572d247 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:38.262356 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d31d5-68c2-44d8-8a2b-11de0572d247 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:38.264178 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 22/47] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:31:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:31:38.269455 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfbea3-7fda-4aee-adbe-138c53565bd9 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:38.269831 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfbea3-7fda-4aee-adbe-138c53565bd9 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:38.270019 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfbea3-7fda-4aee-adbe-138c53565bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:38.287280 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-d4dfbea3-7fda-4aee-adbe-138c53565bd9 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:31:38.288463 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-d4dfbea3-7fda-4aee-adbe-138c53565bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:31:38.358171 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 26/48] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:31:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 13:31:38.380691 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d31d5-68c2-44d8-8a2b-11de0572d247 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:38.440599 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:38.496266 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d31d5-68c2-44d8-8a2b-11de0572d247 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:38.530375 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80726e74-7d3c-4204-9573-a61b7f7cb165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:38.531124 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80726e74-7d3c-4204-9573-a61b7f7cb165 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:38.531414 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80726e74-7d3c-4204-9573-a61b7f7cb165 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:38.531693 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80726e74-7d3c-4204-9573-a61b7f7cb165 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:38.541519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80726e74-7d3c-4204-9573-a61b7f7cb165 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:31:38.550539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-80726e74-7d3c-4204-9573-a61b7f7cb165 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:31:38.550539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:31:38.550539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:31:38.550539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:31:38.550539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:31:38.550539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:31:38.550539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:31:38.550539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:31:38.550539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:31:38.550539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:31:38.550539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:31:38.550539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:31:38.550539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:31:38.550539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:31:38.550539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:31:38.550539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:31:38.551380 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:31:38.551380 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:31:38.551380 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:31:38.551380 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:31:38.551380 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:31:38.551380 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:31:38.551380 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:31:38.551380 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:31:38.551380 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:31:38.551380 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:31:38.551380 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:31:38.551380 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:31:38.551380 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:31:38.551380 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:31:38.551380 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:31:38.551380 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:31:38.552135 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:31:38.552135 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:31:38.552135 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:31:38.552135 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:31:38.552135 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:31:38.552135 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:31:38.552135 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:31:38.552135 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:31:38.552135 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:31:38.552135 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:31:38.552135 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:31:38.552135 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:31:38.552135 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 23/49] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:31:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:31:38.557708 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dfbea3-7fda-4aee-adbe-138c53565bd9 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:38.651881 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dfbea3-7fda-4aee-adbe-138c53565bd9 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:38.659742 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:38.686891 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e470c9-9999-4957-81b7-aa02e85803d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:38.687551 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e470c9-9999-4957-81b7-aa02e85803d4 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:38.687836 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e470c9-9999-4957-81b7-aa02e85803d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:38.688117 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e470c9-9999-4957-81b7-aa02e85803d4 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:38.688866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e470c9-9999-4957-81b7-aa02e85803d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:31:38.689210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e470c9-9999-4957-81b7-aa02e85803d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:38.690602 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e470c9-9999-4957-81b7-aa02e85803d4 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:38.698452 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 27/50] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:31:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:38.704625 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80726e74-7d3c-4204-9573-a61b7f7cb165 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:38.810393 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80726e74-7d3c-4204-9573-a61b7f7cb165 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:38.817955 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:38.850118 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413537d7-cfc8-4f4b-a1c8-7e524734015c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:38.851004 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413537d7-cfc8-4f4b-a1c8-7e524734015c None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:38.851291 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413537d7-cfc8-4f4b-a1c8-7e524734015c None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:38.851573 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413537d7-cfc8-4f4b-a1c8-7e524734015c None admin] PATH_INFO: `/v3/users/c17e13a3d58f4ceab768f46d43e2cab5` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:38.852780 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413537d7-cfc8-4f4b-a1c8-7e524734015c None admin] RBAC: Authorizing `identity:get_user(user_id=c17e13a3d58f4ceab768f46d43e2cab5)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:38.854296 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413537d7-cfc8-4f4b-a1c8-7e524734015c None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:38.855939 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 24/51] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:31:38 2022] GET /identity/v3/users/c17e13a3d58f4ceab768f46d43e2cab5 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:38.862031 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e470c9-9999-4957-81b7-aa02e85803d4 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:38.959022 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e470c9-9999-4957-81b7-aa02e85803d4 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:38.991142 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a816ec5-c3a2-4e0e-8753-d485c4db3970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:38.991928 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a816ec5-c3a2-4e0e-8753-d485c4db3970 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:38.992211 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a816ec5-c3a2-4e0e-8753-d485c4db3970 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:38.992487 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a816ec5-c3a2-4e0e-8753-d485c4db3970 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:38.993641 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a816ec5-c3a2-4e0e-8753-d485c4db3970 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:38.995084 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a816ec5-c3a2-4e0e-8753-d485c4db3970 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:38.996695 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 28/52] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:31:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:39.002528 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413537d7-cfc8-4f4b-a1c8-7e524734015c None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:39.099963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413537d7-cfc8-4f4b-a1c8-7e524734015c None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:39.137231 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0171506-3f4e-4744-bd32-95482710f390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:39.137896 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0171506-3f4e-4744-bd32-95482710f390 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:39.138184 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0171506-3f4e-4744-bd32-95482710f390 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:39.138464 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0171506-3f4e-4744-bd32-95482710f390 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:39.139972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0171506-3f4e-4744-bd32-95482710f390 None admin] RBAC: Adding query filter params (role.id=795daa2c78354fd48e4a741e1182cc98, scope.domain.id=default, user.id=c17e13a3d58f4ceab768f46d43e2cab5) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:31:39.140281 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0171506-3f4e-4744-bd32-95482710f390 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:39.141832 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0171506-3f4e-4744-bd32-95482710f390 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:39.149169 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 25/53] 10.222.0.20 () {56 vars in 1322 bytes} [Wed Jul 6 13:31:39 2022] GET /identity/v3/role_assignments?user.id=c17e13a3d58f4ceab768f46d43e2cab5&scope.domain.id=default&role.id=795daa2c78354fd48e4a741e1182cc98 => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:39.361221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8827b9d-85ec-4f07-9a7d-1579662d556b None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:39.361688 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8827b9d-85ec-4f07-9a7d-1579662d556b None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:39.361783 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8827b9d-85ec-4f07-9a7d-1579662d556b None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:39.363092 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 29/54] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:31:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:31:39.368106 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ea3ef-9378-45f5-b2cd-8e357a8896db None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:39.368499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ea3ef-9378-45f5-b2cd-8e357a8896db None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:39.368680 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ea3ef-9378-45f5-b2cd-8e357a8896db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:39.385543 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-a90ea3ef-9378-45f5-b2cd-8e357a8896db None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:31:39.386733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a90ea3ef-9378-45f5-b2cd-8e357a8896db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:31:39.416127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:39.457343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 26/55] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:31:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:31:39.481177 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8827b9d-85ec-4f07-9a7d-1579662d556b None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:39.518709 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:39.692745 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8827b9d-85ec-4f07-9a7d-1579662d556b None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:39.725064 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c4c99-0ab4-42ea-9b51-00067ccadf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:39.725846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c4c99-0ab4-42ea-9b51-00067ccadf89 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:39.726136 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c4c99-0ab4-42ea-9b51-00067ccadf89 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:39.726420 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c4c99-0ab4-42ea-9b51-00067ccadf89 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:39.727589 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c4c99-0ab4-42ea-9b51-00067ccadf89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:39.729052 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c4c99-0ab4-42ea-9b51-00067ccadf89 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:39.730654 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 30/56] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:31:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:39.737557 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ea3ef-9378-45f5-b2cd-8e357a8896db None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:39.834107 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ea3ef-9378-45f5-b2cd-8e357a8896db None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:39.867157 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf0fe9-83c6-4ff5-a709-bb920283dcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:39.867817 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf0fe9-83c6-4ff5-a709-bb920283dcf9 None admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:39.868106 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf0fe9-83c6-4ff5-a709-bb920283dcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:39.868388 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf0fe9-83c6-4ff5-a709-bb920283dcf9 None admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:39.869125 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf0fe9-83c6-4ff5-a709-bb920283dcf9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:39.870585 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf0fe9-83c6-4ff5-a709-bb920283dcf9 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:39.895044 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 27/57] 10.222.0.20 () {60 vars in 1140 bytes} [Wed Jul 6 13:31:39 2022] POST /identity/v3/projects => generated 298 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:31:41.898987 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3984ce5-d069-48c6-9ba8-edf453aa2fe7 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:41.899497 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3984ce5-d069-48c6-9ba8-edf453aa2fe7 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:41.899607 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3984ce5-d069-48c6-9ba8-edf453aa2fe7 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:41.900915 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 31/58] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:31:41.906735 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72189a-9c5e-4828-8370-a0b7fe1c7e7c None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:41.907134 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72189a-9c5e-4828-8370-a0b7fe1c7e7c None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:41.907302 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72189a-9c5e-4828-8370-a0b7fe1c7e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:41.924857 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-fb72189a-9c5e-4828-8370-a0b7fe1c7e7c None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:31:41.926331 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-fb72189a-9c5e-4828-8370-a0b7fe1c7e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:31:41.943301 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:41.995760 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 28/59] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:31:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:31:42.021268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3984ce5-d069-48c6-9ba8-edf453aa2fe7 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:42.122141 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3984ce5-d069-48c6-9ba8-edf453aa2fe7 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:42.125365 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:42.158578 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0fec61-a452-4da2-b7bd-c07cfa53b981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:42.159282 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fec61-a452-4da2-b7bd-c07cfa53b981 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:42.159569 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fec61-a452-4da2-b7bd-c07cfa53b981 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:42.159845 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fec61-a452-4da2-b7bd-c07cfa53b981 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:42.161074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0fec61-a452-4da2-b7bd-c07cfa53b981 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:42.162505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0fec61-a452-4da2-b7bd-c07cfa53b981 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:42.164124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 32/60] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:31:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:42.169651 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72189a-9c5e-4828-8370-a0b7fe1c7e7c None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:42.266447 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72189a-9c5e-4828-8370-a0b7fe1c7e7c None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:42.299745 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d90af5-5b94-4147-8ef9-09f357089b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:42.300416 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d90af5-5b94-4147-8ef9-09f357089b8f None admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:42.300699 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d90af5-5b94-4147-8ef9-09f357089b8f None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:42.300975 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d90af5-5b94-4147-8ef9-09f357089b8f None admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:42.301717 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d90af5-5b94-4147-8ef9-09f357089b8f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:42.303170 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d90af5-5b94-4147-8ef9-09f357089b8f None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:42.326931 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 29/61] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jul 6 13:31:42 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:31:45.416022 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e668d08-101b-4a2b-bbd7-1a7a66da2d6a None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:45.417173 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e668d08-101b-4a2b-bbd7-1a7a66da2d6a None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:45.417665 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e668d08-101b-4a2b-bbd7-1a7a66da2d6a None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:45.419232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 33/62] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:31:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:31:45.425303 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b19be1-1aac-4415-b5c5-c3e85f0fc093 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:45.426184 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b19be1-1aac-4415-b5c5-c3e85f0fc093 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:45.426667 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b19be1-1aac-4415-b5c5-c3e85f0fc093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:45.441105 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:45.454069 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-45b19be1-1aac-4415-b5c5-c3e85f0fc093 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:31:45.455546 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-45b19be1-1aac-4415-b5c5-c3e85f0fc093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:31:45.496167 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf40e73-2661-415d-a313-0fed4beca78f None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:45.496789 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf40e73-2661-415d-a313-0fed4beca78f None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:45.497314 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf40e73-2661-415d-a313-0fed4beca78f None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:45.499027 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 34/63] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:31:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:31:45.504401 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589d761-8a05-406c-8e41-e87c6203e93f None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:45.504938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589d761-8a05-406c-8e41-e87c6203e93f None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:45.505469 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589d761-8a05-406c-8e41-e87c6203e93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:45.530661 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-1589d761-8a05-406c-8e41-e87c6203e93f None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:31:45.532855 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-1589d761-8a05-406c-8e41-e87c6203e93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:31:45.572133 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 30/64] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:31:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:31:45.596072 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b19be1-1aac-4415-b5c5-c3e85f0fc093 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:45.670376 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 35/65] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:31:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:31:45.696587 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589d761-8a05-406c-8e41-e87c6203e93f None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:45.751976 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b19be1-1aac-4415-b5c5-c3e85f0fc093 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:45.802684 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c630a0e8-8b8d-42da-b108-ae5c79197ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:45.803491 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630a0e8-8b8d-42da-b108-ae5c79197ae4 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:45.803775 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630a0e8-8b8d-42da-b108-ae5c79197ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:45.804053 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630a0e8-8b8d-42da-b108-ae5c79197ae4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:45.809288 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:45.828768 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c630a0e8-8b8d-42da-b108-ae5c79197ae4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:31:45.836414 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-c630a0e8-8b8d-42da-b108-ae5c79197ae4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:31:45.836414 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:31:45.836414 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:31:45.836414 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:31:45.836414 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:31:45.836414 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:31:45.836414 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:31:45.836414 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:31:45.836414 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:31:45.836414 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:31:45.836414 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:31:45.836414 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:31:45.836414 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:31:45.836414 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:31:45.836414 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:31:45.836414 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:31:45.837280 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:31:45.837280 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:31:45.837280 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:31:45.837280 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:31:45.837280 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:31:45.837280 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:31:45.837280 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:31:45.837280 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:31:45.837280 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:31:45.837280 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:31:45.837280 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:31:45.837280 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:31:45.837280 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:31:45.837280 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:31:45.837280 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:31:45.837280 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:31:45.838043 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:31:45.838043 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:31:45.838043 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:31:45.838043 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:31:45.838043 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:31:45.838043 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:31:45.838043 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:31:45.838043 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:31:45.838043 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:31:45.838043 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:31:45.838043 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:31:45.838043 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:31:45.838043 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 31/66] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:31:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:31:45.846495 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c630a0e8-8b8d-42da-b108-ae5c79197ae4 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:45.883923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589d761-8a05-406c-8e41-e87c6203e93f None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:45.886769 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:45.931706 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e822ee0-7ed2-4b9e-b1b6-e2db220894ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:45.932598 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e822ee0-7ed2-4b9e-b1b6-e2db220894ee None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:45.932889 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e822ee0-7ed2-4b9e-b1b6-e2db220894ee None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:45.933210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e822ee0-7ed2-4b9e-b1b6-e2db220894ee None admin] PATH_INFO: `/v3/roles/member` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:45.945226 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e822ee0-7ed2-4b9e-b1b6-e2db220894ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:31:45.953514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-3e822ee0-7ed2-4b9e-b1b6-e2db220894ee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:31:45.953514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:31:45.953514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:31:45.953514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:31:45.953514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:31:45.953514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:31:45.953514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:31:45.953514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:31:45.953514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:31:45.953514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:31:45.953514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:31:45.953514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:31:45.953514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:31:45.953514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:31:45.953514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:31:45.953514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:31:45.954387 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:31:45.954387 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:31:45.954387 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:31:45.954387 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:31:45.954387 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:31:45.954387 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:31:45.954387 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:31:45.954387 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:31:45.954387 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:31:45.954387 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:31:45.954387 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:31:45.954387 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:31:45.954387 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:31:45.954387 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:31:45.954387 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:31:45.954387 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:31:45.955154 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:31:45.955154 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:31:45.955154 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:31:45.955154 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:31:45.955154 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:31:45.955154 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:31:45.955154 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:31:45.955154 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:31:45.955154 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:31:45.955154 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:31:45.955154 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:31:45.955154 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:31:45.955154 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 36/67] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:31:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:31:45.956244 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977f9e5-aaaa-4d9b-92cb-8222c6ef0f53 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:45.956542 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977f9e5-aaaa-4d9b-92cb-8222c6ef0f53 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:45.956827 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977f9e5-aaaa-4d9b-92cb-8222c6ef0f53 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:45.958278 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 37/68] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:31:45.963057 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977f9e5-aaaa-4d9b-92cb-8222c6ef0f53 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:45.999195 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c630a0e8-8b8d-42da-b108-ae5c79197ae4 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:46.042499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a00d935-3ff3-4f4a-b167-3ef0ba053120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:46.043342 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a00d935-3ff3-4f4a-b167-3ef0ba053120 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:46.043732 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a00d935-3ff3-4f4a-b167-3ef0ba053120 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:46.044117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a00d935-3ff3-4f4a-b167-3ef0ba053120 None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:46.045526 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a00d935-3ff3-4f4a-b167-3ef0ba053120 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:31:46.045950 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a00d935-3ff3-4f4a-b167-3ef0ba053120 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:46.047488 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a00d935-3ff3-4f4a-b167-3ef0ba053120 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:46.059924 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 32/69] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:31:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:46.063202 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e059c-55b6-4ae1-abc5-86d9ecf03396 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:46.063494 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e059c-55b6-4ae1-abc5-86d9ecf03396 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:46.063775 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e059c-55b6-4ae1-abc5-86d9ecf03396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:46.083497 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977f9e5-aaaa-4d9b-92cb-8222c6ef0f53 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:46.089206 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-a84e059c-55b6-4ae1-abc5-86d9ecf03396 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:31:46.090418 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a84e059c-55b6-4ae1-abc5-86d9ecf03396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:31:46.120159 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593037c-a953-4bcb-a50e-89bcca973907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:46.120831 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593037c-a953-4bcb-a50e-89bcca973907 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:46.121151 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593037c-a953-4bcb-a50e-89bcca973907 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:46.121434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593037c-a953-4bcb-a50e-89bcca973907 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:46.122183 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c593037c-a953-4bcb-a50e-89bcca973907 None admin] RBAC: Adding query filter params (name=member) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:31:46.122495 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c593037c-a953-4bcb-a50e-89bcca973907 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:46.123933 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c593037c-a953-4bcb-a50e-89bcca973907 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:46.132560 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 38/70] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jul 6 13:31:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:46.137258 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593037c-a953-4bcb-a50e-89bcca973907 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:46.153336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:46.160129 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:46.170539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 33/71] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:31:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:31:46.171839 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc72d73-baf4-4e0f-8fdd-ea72463e71bc None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:46.172129 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc72d73-baf4-4e0f-8fdd-ea72463e71bc None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:46.172410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc72d73-baf4-4e0f-8fdd-ea72463e71bc None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:46.173859 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 34/72] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:31:46.179140 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf4b26-3953-4916-b251-93bee9250f77 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:46.179440 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf4b26-3953-4916-b251-93bee9250f77 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:46.179723 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf4b26-3953-4916-b251-93bee9250f77 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:46.181141 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 35/73] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:31:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:31:46.181671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf4b26-3953-4916-b251-93bee9250f77 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:46.236096 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593037c-a953-4bcb-a50e-89bcca973907 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:46.269420 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d05348-5a04-4d44-b9e9-e490dc6a754e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:46.270308 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d05348-5a04-4d44-b9e9-e490dc6a754e None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:46.270589 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d05348-5a04-4d44-b9e9-e490dc6a754e None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:46.270867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d05348-5a04-4d44-b9e9-e490dc6a754e None admin] PATH_INFO: `/v3/users/c17e13a3d58f4ceab768f46d43e2cab5` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:46.272061 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d05348-5a04-4d44-b9e9-e490dc6a754e None admin] RBAC: Authorizing `identity:get_user(user_id=c17e13a3d58f4ceab768f46d43e2cab5)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:46.273542 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d05348-5a04-4d44-b9e9-e490dc6a754e None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:46.275330 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 39/74] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:31:46 2022] GET /identity/v3/users/c17e13a3d58f4ceab768f46d43e2cab5 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:46.276411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da46a49-c01d-4483-ab15-6bab088875c1 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:46.276703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da46a49-c01d-4483-ab15-6bab088875c1 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:46.276980 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da46a49-c01d-4483-ab15-6bab088875c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:46.286465 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf4b26-3953-4916-b251-93bee9250f77 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:46.292790 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-4da46a49-c01d-4483-ab15-6bab088875c1 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:31:46.294191 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-4da46a49-c01d-4483-ab15-6bab088875c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:31:46.321452 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1511b3-e5f3-449d-b25c-7e5b2fd804d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:46.322327 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1511b3-e5f3-449d-b25c-7e5b2fd804d7 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:46.322611 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1511b3-e5f3-449d-b25c-7e5b2fd804d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:46.322891 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1511b3-e5f3-449d-b25c-7e5b2fd804d7 None admin] PATH_INFO: `/v3/users/31e4ee38bf8148f4a4dd9f56097514a1` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:46.336689 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1511b3-e5f3-449d-b25c-7e5b2fd804d7 None admin] RBAC: Authorizing `identity:get_user(user_id=31e4ee38bf8148f4a4dd9f56097514a1)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:46.338114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1511b3-e5f3-449d-b25c-7e5b2fd804d7 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:46.339761 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 36/75] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:31:46 2022] GET /identity/v3/users/31e4ee38bf8148f4a4dd9f56097514a1 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:46.340993 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c02ba-85f1-40f3-8c1f-7417edf9123a None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:46.341308 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c02ba-85f1-40f3-8c1f-7417edf9123a None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:46.341834 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c02ba-85f1-40f3-8c1f-7417edf9123a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:46.358259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-e50c02ba-85f1-40f3-8c1f-7417edf9123a None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:31:46.359457 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-e50c02ba-85f1-40f3-8c1f-7417edf9123a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:31:46.366870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 40/76] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:31:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:31:46.367706 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da46a49-c01d-4483-ab15-6bab088875c1 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:46.391431 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:46.404359 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:46.429818 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 37/77] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:31:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:31:46.430714 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50c02ba-85f1-40f3-8c1f-7417edf9123a None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:46.469211 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da46a49-c01d-4483-ab15-6bab088875c1 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:46.504125 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410878f8-fe2a-4904-98f7-977d81653c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:46.504941 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410878f8-fe2a-4904-98f7-977d81653c9e None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:46.505261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410878f8-fe2a-4904-98f7-977d81653c9e None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:46.505547 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410878f8-fe2a-4904-98f7-977d81653c9e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:46.516116 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410878f8-fe2a-4904-98f7-977d81653c9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:31:46.524137 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-410878f8-fe2a-4904-98f7-977d81653c9e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 13:31:46.524137 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:31:46.524137 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:31:46.524137 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:31:46.524137 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:31:46.524137 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:31:46.524137 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:31:46.524137 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:31:46.524137 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:31:46.524137 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:31:46.524137 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:31:46.524137 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:31:46.524137 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:31:46.524137 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:31:46.524137 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:31:46.524137 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:31:46.525048 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:31:46.525048 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:31:46.525048 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:31:46.525048 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:31:46.525048 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:31:46.525048 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:31:46.525048 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:31:46.525048 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:31:46.525048 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:31:46.525048 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:31:46.525048 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:31:46.525048 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:31:46.525048 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:31:46.525048 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:31:46.525048 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:31:46.525048 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:31:46.525821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:31:46.525821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:31:46.525821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:31:46.525821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:31:46.525821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:31:46.525821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:31:46.525821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:31:46.525821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:31:46.525821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:31:46.525821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:31:46.525821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 13:31:46.525821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:31:46.525821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 41/78] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jul 6 13:31:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:31:46.526440 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410878f8-fe2a-4904-98f7-977d81653c9e None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:46.530414 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50c02ba-85f1-40f3-8c1f-7417edf9123a None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:46.564117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d981b2d-c1a0-4ac6-beb6-72f45fa97be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:46.564773 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d981b2d-c1a0-4ac6-beb6-72f45fa97be8 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:46.565084 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d981b2d-c1a0-4ac6-beb6-72f45fa97be8 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:46.565365 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d981b2d-c1a0-4ac6-beb6-72f45fa97be8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:46.608745 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:46.609713 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:46.628313 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410878f8-fe2a-4904-98f7-977d81653c9e None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:46.632369 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d981b2d-c1a0-4ac6-beb6-72f45fa97be8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:46.633877 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d981b2d-c1a0-4ac6-beb6-72f45fa97be8 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:46.661426 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3128941b-58c6-4716-8014-d99bebfc1f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:46.662068 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128941b-58c6-4716-8014-d99bebfc1f01 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:46.662355 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128941b-58c6-4716-8014-d99bebfc1f01 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:46.662636 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128941b-58c6-4716-8014-d99bebfc1f01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:46.728238 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3128941b-58c6-4716-8014-d99bebfc1f01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:46.729762 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3128941b-58c6-4716-8014-d99bebfc1f01 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:46.731026 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 38/79] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:31:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:31:46.731871 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d981b2d-c1a0-4ac6-beb6-72f45fa97be8 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:46.809708 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:46.811064 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:46.830727 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 42/80] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:31:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:31:46.833174 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3128941b-58c6-4716-8014-d99bebfc1f01 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:46.838750 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d981b2d-c1a0-4ac6-beb6-72f45fa97be8 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:46.873960 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486cad7-6b95-4b92-9618-42483a53824e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:46.874497 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486cad7-6b95-4b92-9618-42483a53824e None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:46.874779 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486cad7-6b95-4b92-9618-42483a53824e None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:46.875053 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486cad7-6b95-4b92-9618-42483a53824e None admin] PATH_INFO: `/v3/roles/member` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:46.884832 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5486cad7-6b95-4b92-9618-42483a53824e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:31:46.892564 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-5486cad7-6b95-4b92-9618-42483a53824e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:31:46.892564 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:31:46.892564 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:31:46.892564 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:31:46.892564 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:31:46.892564 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:31:46.892564 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:31:46.892564 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:31:46.892564 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:31:46.892564 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:31:46.892564 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:31:46.892564 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:31:46.892564 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:31:46.892564 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:31:46.892564 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:31:46.892564 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:31:46.893458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:31:46.893458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:31:46.893458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:31:46.893458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:31:46.893458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:31:46.893458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:31:46.893458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:31:46.893458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:31:46.893458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:31:46.893458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:31:46.893458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:31:46.893458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:31:46.893458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:31:46.893458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:31:46.893458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:31:46.893458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:31:46.894228 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:31:46.894228 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:31:46.894228 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:31:46.894228 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:31:46.894228 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:31:46.894228 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:31:46.894228 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:31:46.894228 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:31:46.894228 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:31:46.894228 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:31:46.894228 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:31:46.894228 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:31:46.894228 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 39/81] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:31:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:31:46.894851 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486cad7-6b95-4b92-9618-42483a53824e None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:46.929546 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3128941b-58c6-4716-8014-d99bebfc1f01 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:46.963036 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63496a43-8c00-4996-b84e-ca59b9fb8396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:46.963832 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63496a43-8c00-4996-b84e-ca59b9fb8396 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:46.964119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63496a43-8c00-4996-b84e-ca59b9fb8396 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:46.964397 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63496a43-8c00-4996-b84e-ca59b9fb8396 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:46.965599 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63496a43-8c00-4996-b84e-ca59b9fb8396 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:46.967855 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63496a43-8c00-4996-b84e-ca59b9fb8396 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:46.969511 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 43/82] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:31:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:46.970272 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63496a43-8c00-4996-b84e-ca59b9fb8396 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:46.994152 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486cad7-6b95-4b92-9618-42483a53824e None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:47.018781 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:47.026695 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc32d9d-fde2-4272-bb0f-941566e3755a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:47.027363 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc32d9d-fde2-4272-bb0f-941566e3755a None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:47.027649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc32d9d-fde2-4272-bb0f-941566e3755a None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:47.027930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc32d9d-fde2-4272-bb0f-941566e3755a None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:47.028676 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc32d9d-fde2-4272-bb0f-941566e3755a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:31:47.029136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc32d9d-fde2-4272-bb0f-941566e3755a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:47.030617 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc32d9d-fde2-4272-bb0f-941566e3755a None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:47.032825 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:47.039584 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 40/83] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:31:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:47.040392 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc32d9d-fde2-4272-bb0f-941566e3755a None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:47.069597 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63496a43-8c00-4996-b84e-ca59b9fb8396 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:47.103024 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09cacd-4405-41ba-bdc2-5539df9608b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:47.103909 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09cacd-4405-41ba-bdc2-5539df9608b8 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:47.104197 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09cacd-4405-41ba-bdc2-5539df9608b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:47.104478 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09cacd-4405-41ba-bdc2-5539df9608b8 None admin] PATH_INFO: `/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:47.105698 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef09cacd-4405-41ba-bdc2-5539df9608b8 None admin] RBAC: Authorizing `identity:get_project(project_id=202d7ba5c7f34872aa3ab82e5d2b2f93)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:47.107659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef09cacd-4405-41ba-bdc2-5539df9608b8 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:47.109511 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 44/84] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:31:46 2022] GET /identity/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:47.115035 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09cacd-4405-41ba-bdc2-5539df9608b8 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:47.139305 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc32d9d-fde2-4272-bb0f-941566e3755a None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:47.171753 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60dcba4-3378-42a2-9910-a2053190b792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:47.172616 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60dcba4-3378-42a2-9910-a2053190b792 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:47.172903 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60dcba4-3378-42a2-9910-a2053190b792 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:47.173207 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60dcba4-3378-42a2-9910-a2053190b792 None admin] PATH_INFO: `/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:47.174639 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60dcba4-3378-42a2-9910-a2053190b792 None admin] RBAC: Authorizing `identity:get_project(project_id=202d7ba5c7f34872aa3ab82e5d2b2f93)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:47.176075 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60dcba4-3378-42a2-9910-a2053190b792 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:47.177796 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 41/85] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:31:47 2022] GET /identity/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93 => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:47.178530 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60dcba4-3378-42a2-9910-a2053190b792 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:47.215489 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09cacd-4405-41ba-bdc2-5539df9608b8 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:47.232645 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:47.241373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:47.249926 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c53d09-036a-4d45-a220-475adf3be3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:47.250594 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c53d09-036a-4d45-a220-475adf3be3bb None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:47.250880 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c53d09-036a-4d45-a220-475adf3be3bb None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:47.251158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c53d09-036a-4d45-a220-475adf3be3bb None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:47.251908 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c53d09-036a-4d45-a220-475adf3be3bb None admin] RBAC: Adding query filter params (name=member) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:31:47.252215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c53d09-036a-4d45-a220-475adf3be3bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:47.253608 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c53d09-036a-4d45-a220-475adf3be3bb None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:47.261899 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 45/86] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jul 6 13:31:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:47.262696 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c53d09-036a-4d45-a220-475adf3be3bb None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:47.277871 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60dcba4-3378-42a2-9910-a2053190b792 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:47.312642 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c935d6-a523-42c4-905b-819abde28fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:47.313353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c935d6-a523-42c4-905b-819abde28fa2 None admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:47.313708 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c935d6-a523-42c4-905b-819abde28fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:47.314002 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c935d6-a523-42c4-905b-819abde28fa2 None admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:47.314728 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c935d6-a523-42c4-905b-819abde28fa2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:47.316195 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c935d6-a523-42c4-905b-819abde28fa2 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:47.336127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 42/87] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jul 6 13:31:47 2022] POST /identity/v3/users => generated 289 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:31:47.337114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c935d6-a523-42c4-905b-819abde28fa2 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:47.374461 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c53d09-036a-4d45-a220-475adf3be3bb None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:47.417290 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb7418-181c-4219-9b19-eaf899c23b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:47.418098 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb7418-181c-4219-9b19-eaf899c23b73 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:47.418392 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb7418-181c-4219-9b19-eaf899c23b73 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:47.418680 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb7418-181c-4219-9b19-eaf899c23b73 None admin] PATH_INFO: `/v3/users/31e4ee38bf8148f4a4dd9f56097514a1` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:47.420085 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feb7418-181c-4219-9b19-eaf899c23b73 None admin] RBAC: Authorizing `identity:get_user(user_id=31e4ee38bf8148f4a4dd9f56097514a1)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:47.421604 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feb7418-181c-4219-9b19-eaf899c23b73 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:47.423308 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 46/88] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:31:47 2022] GET /identity/v3/users/31e4ee38bf8148f4a4dd9f56097514a1 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:47.424352 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb7418-181c-4219-9b19-eaf899c23b73 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:47.465249 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c935d6-a523-42c4-905b-819abde28fa2 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:47.480828 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:47.491977 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:47.510377 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25b2d2e-b004-4ca3-9b44-72245ff70ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:47.511044 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b2d2e-b004-4ca3-9b44-72245ff70ce3 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:47.511327 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b2d2e-b004-4ca3-9b44-72245ff70ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:47.512035 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b2d2e-b004-4ca3-9b44-72245ff70ce3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:47.512887 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25b2d2e-b004-4ca3-9b44-72245ff70ce3 None admin] RBAC: Adding query filter params (role.id=795daa2c78354fd48e4a741e1182cc98, scope.project.id=202d7ba5c7f34872aa3ab82e5d2b2f93, user.id=c17e13a3d58f4ceab768f46d43e2cab5) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:31:47.513307 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25b2d2e-b004-4ca3-9b44-72245ff70ce3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:47.516058 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25b2d2e-b004-4ca3-9b44-72245ff70ce3 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:47.522455 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 43/89] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:31:47 2022] GET /identity/v3/role_assignments?user.id=c17e13a3d58f4ceab768f46d43e2cab5&scope.project.id=202d7ba5c7f34872aa3ab82e5d2b2f93&role.id=795daa2c78354fd48e4a741e1182cc98 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:47.523257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25b2d2e-b004-4ca3-9b44-72245ff70ce3 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:47.554039 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb7418-181c-4219-9b19-eaf899c23b73 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:47.605970 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850124e-46dc-43eb-b8d7-24434c7f832b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:47.607408 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850124e-46dc-43eb-b8d7-24434c7f832b None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:47.607408 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850124e-46dc-43eb-b8d7-24434c7f832b None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:47.607900 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850124e-46dc-43eb-b8d7-24434c7f832b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:47.608791 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850124e-46dc-43eb-b8d7-24434c7f832b None admin] RBAC: Adding query filter params (role.id=b3244d1d9f004c1b8d3594fd30da1024, scope.project.id=202d7ba5c7f34872aa3ab82e5d2b2f93, user.id=31e4ee38bf8148f4a4dd9f56097514a1) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:31:47.609194 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850124e-46dc-43eb-b8d7-24434c7f832b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:47.611439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850124e-46dc-43eb-b8d7-24434c7f832b None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:47.618732 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 47/90] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:31:47 2022] GET /identity/v3/role_assignments?user.id=31e4ee38bf8148f4a4dd9f56097514a1&scope.project.id=202d7ba5c7f34872aa3ab82e5d2b2f93&role.id=b3244d1d9f004c1b8d3594fd30da1024 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:47.620168 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850124e-46dc-43eb-b8d7-24434c7f832b None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:47.663455 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25b2d2e-b004-4ca3-9b44-72245ff70ce3 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:47.716284 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc6901-e086-41b3-90f0-d400d69cce0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:47.719204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc6901-e086-41b3-90f0-d400d69cce0d None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:47.719489 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc6901-e086-41b3-90f0-d400d69cce0d None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:47.719773 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc6901-e086-41b3-90f0-d400d69cce0d None admin] PATH_INFO: `/v3/users/31e4ee38bf8148f4a4dd9f56097514a1` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:47.721860 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc6901-e086-41b3-90f0-d400d69cce0d None admin] RBAC: Authorizing `identity:get_user(user_id=31e4ee38bf8148f4a4dd9f56097514a1)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:47.723389 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc6901-e086-41b3-90f0-d400d69cce0d None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:47.725178 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 44/91] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:31:47 2022] GET /identity/v3/users/31e4ee38bf8148f4a4dd9f56097514a1 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:47.730773 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc6901-e086-41b3-90f0-d400d69cce0d None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:47.774388 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850124e-46dc-43eb-b8d7-24434c7f832b None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:47.791283 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:47.807608 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:47.819769 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe340b-4421-418b-aae5-200aa6fd19e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:47.820543 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe340b-4421-418b-aae5-200aa6fd19e1 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:47.820897 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe340b-4421-418b-aae5-200aa6fd19e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:47.821279 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe340b-4421-418b-aae5-200aa6fd19e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:47.869930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc6901-e086-41b3-90f0-d400d69cce0d None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:47.902421 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe340b-4421-418b-aae5-200aa6fd19e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:47.904220 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe340b-4421-418b-aae5-200aa6fd19e1 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:47.915851 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc0eba-5b67-426f-adb9-0357820a876f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:47.916836 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc0eba-5b67-426f-adb9-0357820a876f None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:47.917743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc0eba-5b67-426f-adb9-0357820a876f None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:47.918227 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc0eba-5b67-426f-adb9-0357820a876f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:47.999604 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc0eba-5b67-426f-adb9-0357820a876f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:48.001534 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc0eba-5b67-426f-adb9-0357820a876f None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:48.024976 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 48/92] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:31:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:31:48.030887 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe340b-4421-418b-aae5-200aa6fd19e1 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:48.045928 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:48.059198 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:48.108425 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 45/93] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:31:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:31:48.113950 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc0eba-5b67-426f-adb9-0357820a876f None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:48.148109 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe340b-4421-418b-aae5-200aa6fd19e1 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:48.186844 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8575f6d-35b7-426f-886a-211bf06a4bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:48.187711 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8575f6d-35b7-426f-886a-211bf06a4bd1 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:48.188024 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8575f6d-35b7-426f-886a-211bf06a4bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:48.188315 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8575f6d-35b7-426f-886a-211bf06a4bd1 None admin] PATH_INFO: `/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:48.189708 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8575f6d-35b7-426f-886a-211bf06a4bd1 None admin] RBAC: Authorizing `identity:get_project(project_id=202d7ba5c7f34872aa3ab82e5d2b2f93)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:48.191510 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8575f6d-35b7-426f-886a-211bf06a4bd1 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:48.193554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 49/94] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:31:48 2022] GET /identity/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:48.199310 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8575f6d-35b7-426f-886a-211bf06a4bd1 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:48.302353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc0eba-5b67-426f-adb9-0357820a876f None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:48.313983 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:48.336220 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8575f6d-35b7-426f-886a-211bf06a4bd1 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:48.346096 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb7574-8653-4deb-9bd2-179f3335ca17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:48.347301 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb7574-8653-4deb-9bd2-179f3335ca17 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:48.347301 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb7574-8653-4deb-9bd2-179f3335ca17 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:48.347552 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb7574-8653-4deb-9bd2-179f3335ca17 None admin] PATH_INFO: `/v3/projects/a3eb71934e924f9e801e61e599cba492` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:48.348755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb7574-8653-4deb-9bd2-179f3335ca17 None admin] RBAC: Authorizing `identity:get_project(project_id=a3eb71934e924f9e801e61e599cba492)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:48.350569 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb7574-8653-4deb-9bd2-179f3335ca17 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:48.352358 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 46/95] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:31:48 2022] GET /identity/v3/projects/a3eb71934e924f9e801e61e599cba492 => generated 312 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:48.358217 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb7574-8653-4deb-9bd2-179f3335ca17 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:48.400026 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a65ea-6d42-49c5-b20f-a7e4b5aa4c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:48.400692 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a65ea-6d42-49c5-b20f-a7e4b5aa4c74 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:48.400974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a65ea-6d42-49c5-b20f-a7e4b5aa4c74 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:48.401333 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a65ea-6d42-49c5-b20f-a7e4b5aa4c74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:48.402178 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a65ea-6d42-49c5-b20f-a7e4b5aa4c74 None admin] RBAC: Adding query filter params (role.id=534400e7b28546a8a04c08f825c28566, scope.project.id=202d7ba5c7f34872aa3ab82e5d2b2f93, user.id=31e4ee38bf8148f4a4dd9f56097514a1) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:31:48.402542 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a65ea-6d42-49c5-b20f-a7e4b5aa4c74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:48.404087 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a65ea-6d42-49c5-b20f-a7e4b5aa4c74 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:48.422378 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 50/96] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:31:48 2022] GET /identity/v3/role_assignments?user.id=31e4ee38bf8148f4a4dd9f56097514a1&scope.project.id=202d7ba5c7f34872aa3ab82e5d2b2f93&role.id=534400e7b28546a8a04c08f825c28566 => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:31:48.424192 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:48.555317 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb7574-8653-4deb-9bd2-179f3335ca17 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:48.613866 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b362f5b-ccea-4b6a-ab8e-2cecb6bf0113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:48.614563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b362f5b-ccea-4b6a-ab8e-2cecb6bf0113 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:48.614855 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b362f5b-ccea-4b6a-ab8e-2cecb6bf0113 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:48.615140 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b362f5b-ccea-4b6a-ab8e-2cecb6bf0113 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:48.616005 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b362f5b-ccea-4b6a-ab8e-2cecb6bf0113 None admin] RBAC: Adding query filter params (role.id=b3244d1d9f004c1b8d3594fd30da1024, scope.project.id=a3eb71934e924f9e801e61e599cba492, user.id=31e4ee38bf8148f4a4dd9f56097514a1) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:31:48.616323 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b362f5b-ccea-4b6a-ab8e-2cecb6bf0113 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:48.617830 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b362f5b-ccea-4b6a-ab8e-2cecb6bf0113 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:48.624950 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 47/97] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:31:48 2022] GET /identity/v3/role_assignments?user.id=31e4ee38bf8148f4a4dd9f56097514a1&scope.project.id=a3eb71934e924f9e801e61e599cba492&role.id=b3244d1d9f004c1b8d3594fd30da1024 => generated 259 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:50.279807 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c69586-beff-481c-8af1-dd0084d38b08 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:50.280343 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c69586-beff-481c-8af1-dd0084d38b08 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:50.280343 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c69586-beff-481c-8af1-dd0084d38b08 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:50.281755 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 51/98] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:31:50.287231 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7502d-9d52-4dea-b5ef-f9fdc2646f2c None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:50.287625 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7502d-9d52-4dea-b5ef-f9fdc2646f2c None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:50.287789 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7502d-9d52-4dea-b5ef-f9fdc2646f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:50.315318 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-d3c7502d-9d52-4dea-b5ef-f9fdc2646f2c None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:31:50.316533 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-d3c7502d-9d52-4dea-b5ef-f9fdc2646f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:31:50.403350 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 48/99] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:31:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:31:50.439391 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c69586-beff-481c-8af1-dd0084d38b08 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:50.504679 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a5bf2-521a-4c74-98f6-31ee6570a41c None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:50.504974 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a5bf2-521a-4c74-98f6-31ee6570a41c None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:50.505288 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a5bf2-521a-4c74-98f6-31ee6570a41c None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:50.506850 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 49/100] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:31:50.512170 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339fe62f-a788-486d-bf79-5aed5ac2aff0 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:50.512468 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339fe62f-a788-486d-bf79-5aed5ac2aff0 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:50.512753 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339fe62f-a788-486d-bf79-5aed5ac2aff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:50.546871 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-339fe62f-a788-486d-bf79-5aed5ac2aff0 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:31:50.547963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-339fe62f-a788-486d-bf79-5aed5ac2aff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:31:50.578644 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:50.598603 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c69586-beff-481c-8af1-dd0084d38b08 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:50.651092 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755d1f0c-1f31-4495-8f3d-6a3f910b3125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:50.651910 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d1f0c-1f31-4495-8f3d-6a3f910b3125 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:50.652195 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d1f0c-1f31-4495-8f3d-6a3f910b3125 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:50.652475 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d1f0c-1f31-4495-8f3d-6a3f910b3125 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:50.671591 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755d1f0c-1f31-4495-8f3d-6a3f910b3125 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:31:50.684605 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-755d1f0c-1f31-4495-8f3d-6a3f910b3125 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:31:50.684605 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:31:50.684605 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:31:50.684605 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:31:50.684605 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:31:50.684605 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:31:50.684605 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:31:50.684605 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:31:50.684605 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:31:50.684605 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:31:50.684605 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:31:50.684605 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:31:50.684605 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:31:50.684605 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:31:50.684605 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:31:50.684605 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:31:50.685866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:31:50.685866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:31:50.685866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:31:50.685866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:31:50.685866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:31:50.685866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:31:50.685866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:31:50.685866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:31:50.685866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:31:50.685866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:31:50.685866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:31:50.685866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:31:50.685866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:31:50.685866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:31:50.685866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:31:50.685866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:31:50.686802 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:31:50.686802 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:31:50.686802 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:31:50.686802 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:31:50.686802 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:31:50.686802 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:31:50.686802 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:31:50.686802 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:31:50.686802 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:31:50.686802 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:31:50.686802 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:31:50.686802 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:31:50.686802 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 52/101] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:31:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:31:50.687471 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ada8e2-b42f-4af3-a105-02dd65acb07c None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:50.688328 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ada8e2-b42f-4af3-a105-02dd65acb07c None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:50.688328 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ada8e2-b42f-4af3-a105-02dd65acb07c None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:50.693884 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 53/102] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:31:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:31:50.694909 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ada8e2-b42f-4af3-a105-02dd65acb07c None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:50.702981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:50.714672 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 50/103] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:31:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 13:31:50.717056 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f63fbf-3acb-4cc3-8448-1297ba69d3e6 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:50.717732 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f63fbf-3acb-4cc3-8448-1297ba69d3e6 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:50.718223 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f63fbf-3acb-4cc3-8448-1297ba69d3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:50.743993 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-e9f63fbf-3acb-4cc3-8448-1297ba69d3e6 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:31:50.745550 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-e9f63fbf-3acb-4cc3-8448-1297ba69d3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:31:50.852251 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ada8e2-b42f-4af3-a105-02dd65acb07c None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:50.890447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4531e-f69f-4504-9f71-f9ee44339842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:50.891601 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4531e-f69f-4504-9f71-f9ee44339842 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:50.892096 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4531e-f69f-4504-9f71-f9ee44339842 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:50.892580 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4531e-f69f-4504-9f71-f9ee44339842 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:50.893629 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da4531e-f69f-4504-9f71-f9ee44339842 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:31:50.894165 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da4531e-f69f-4504-9f71-f9ee44339842 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:50.894892 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 51/104] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:31:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:31:50.895746 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f63fbf-3acb-4cc3-8448-1297ba69d3e6 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:50.896324 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da4531e-f69f-4504-9f71-f9ee44339842 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:50.907744 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 54/105] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:31:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:50.918096 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4531e-f69f-4504-9f71-f9ee44339842 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:51.010078 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:51.043172 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:51.051253 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4531e-f69f-4504-9f71-f9ee44339842 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:51.091458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f63fbf-3acb-4cc3-8448-1297ba69d3e6 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:51.091925 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c1a8e1-8963-4250-a53c-ec3fed7521e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:51.093349 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1a8e1-8963-4250-a53c-ec3fed7521e7 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:51.093870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1a8e1-8963-4250-a53c-ec3fed7521e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:51.094342 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1a8e1-8963-4250-a53c-ec3fed7521e7 None admin] PATH_INFO: `/v3/users/c17e13a3d58f4ceab768f46d43e2cab5` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:51.095771 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c1a8e1-8963-4250-a53c-ec3fed7521e7 None admin] RBAC: Authorizing `identity:get_user(user_id=c17e13a3d58f4ceab768f46d43e2cab5)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:51.098273 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c1a8e1-8963-4250-a53c-ec3fed7521e7 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:51.100161 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 55/106] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:31:50 2022] GET /identity/v3/users/c17e13a3d58f4ceab768f46d43e2cab5 => generated 247 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:51.101267 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c1a8e1-8963-4250-a53c-ec3fed7521e7 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:51.167802 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076e1aa-1649-4240-b855-965974e43a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:51.168538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076e1aa-1649-4240-b855-965974e43a28 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:51.168878 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076e1aa-1649-4240-b855-965974e43a28 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:51.169197 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076e1aa-1649-4240-b855-965974e43a28 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:51.195651 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d076e1aa-1649-4240-b855-965974e43a28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:31:51.210821 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-d076e1aa-1649-4240-b855-965974e43a28 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:31:51.210821 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:31:51.210821 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:31:51.210821 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:31:51.210821 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:31:51.210821 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:31:51.210821 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:31:51.210821 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:31:51.210821 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:31:51.210821 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:31:51.210821 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:31:51.210821 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:31:51.210821 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:31:51.210821 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:31:51.210821 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:31:51.210821 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:31:51.211733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:31:51.211733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:31:51.211733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:31:51.211733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:31:51.211733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:31:51.211733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:31:51.211733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:31:51.211733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:31:51.211733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:31:51.211733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:31:51.211733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:31:51.211733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:31:51.211733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:31:51.211733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:31:51.211733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:31:51.211733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:31:51.212923 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:31:51.212923 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:31:51.212923 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:31:51.212923 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:31:51.212923 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:31:51.212923 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:31:51.212923 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:31:51.212923 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:31:51.212923 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:31:51.212923 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:31:51.212923 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:31:51.212923 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:31:51.212923 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 52/107] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:31:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:31:51.213817 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076e1aa-1649-4240-b855-965974e43a28 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:51.219339 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c1a8e1-8963-4250-a53c-ec3fed7521e7 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:51.261286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb086d20-bc84-4797-bc0d-59bf3a9db31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:51.262680 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb086d20-bc84-4797-bc0d-59bf3a9db31d None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:51.263406 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb086d20-bc84-4797-bc0d-59bf3a9db31d None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:51.264379 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb086d20-bc84-4797-bc0d-59bf3a9db31d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:51.266984 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:51.282662 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb086d20-bc84-4797-bc0d-59bf3a9db31d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:31:51.293086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-fb086d20-bc84-4797-bc0d-59bf3a9db31d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 13:31:51.293086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:31:51.293086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:31:51.293086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:31:51.293086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:31:51.293086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:31:51.293086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:31:51.293086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:31:51.293086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:31:51.293086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:31:51.293086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:31:51.293086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:31:51.293086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:31:51.293086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:31:51.293086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:31:51.293086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:31:51.294118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:31:51.294118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:31:51.294118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:31:51.294118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:31:51.294118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:31:51.294118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:31:51.294118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:31:51.294118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:31:51.294118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:31:51.294118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:31:51.294118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:31:51.294118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:31:51.294118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:31:51.294118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:31:51.294118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:31:51.294118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:31:51.295405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:31:51.295405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:31:51.295405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:31:51.295405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:31:51.295405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:31:51.295405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:31:51.295405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:31:51.295405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:31:51.295405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:31:51.295405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:31:51.295405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 13:31:51.295405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:31:51.298554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 56/108] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:31:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:31:51.300268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb086d20-bc84-4797-bc0d-59bf3a9db31d None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:51.350568 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076e1aa-1649-4240-b855-965974e43a28 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:51.388240 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0c3a8-caea-4555-a9da-a0f98602bc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:51.389781 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0c3a8-caea-4555-a9da-a0f98602bc53 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:51.390068 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0c3a8-caea-4555-a9da-a0f98602bc53 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:51.390349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0c3a8-caea-4555-a9da-a0f98602bc53 None admin] PATH_INFO: `/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:51.391576 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0c3a8-caea-4555-a9da-a0f98602bc53 None admin] RBAC: Authorizing `identity:get_project(project_id=202d7ba5c7f34872aa3ab82e5d2b2f93)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:51.393160 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0c3a8-caea-4555-a9da-a0f98602bc53 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:51.395112 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 53/109] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:31:51 2022] GET /identity/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:51.396078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237f517-3bb4-4f06-bdf3-28ead3d708af None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:51.396365 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237f517-3bb4-4f06-bdf3-28ead3d708af None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:51.396641 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237f517-3bb4-4f06-bdf3-28ead3d708af None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:51.398058 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 54/110] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:31:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:31:51.402540 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237f517-3bb4-4f06-bdf3-28ead3d708af None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:51.406723 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:51.423600 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb086d20-bc84-4797-bc0d-59bf3a9db31d None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:51.459316 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f74ecc-f90f-4a2c-87c1-8d17d5c1e5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:51.459993 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f74ecc-f90f-4a2c-87c1-8d17d5c1e5dd None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:51.460280 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f74ecc-f90f-4a2c-87c1-8d17d5c1e5dd None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:51.460563 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f74ecc-f90f-4a2c-87c1-8d17d5c1e5dd None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:51.461344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f74ecc-f90f-4a2c-87c1-8d17d5c1e5dd None admin] RBAC: Adding query filter params (name=member) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:31:51.461655 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f74ecc-f90f-4a2c-87c1-8d17d5c1e5dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:51.463032 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f74ecc-f90f-4a2c-87c1-8d17d5c1e5dd None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:51.473255 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 57/111] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jul 6 13:31:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:51.474462 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45d9df-8380-436a-af63-431fd51e627b None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:51.474751 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45d9df-8380-436a-af63-431fd51e627b None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:51.475030 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45d9df-8380-436a-af63-431fd51e627b None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:51.476407 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 58/112] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:31:51.476928 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45d9df-8380-436a-af63-431fd51e627b None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:51.518651 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237f517-3bb4-4f06-bdf3-28ead3d708af None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:51.544727 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:51.555070 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3f759-6163-4729-9ad3-2adc536ee3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:51.555743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3f759-6163-4729-9ad3-2adc536ee3fc None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:51.556030 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3f759-6163-4729-9ad3-2adc536ee3fc None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:51.556306 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3f759-6163-4729-9ad3-2adc536ee3fc None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:51.557087 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3f759-6163-4729-9ad3-2adc536ee3fc None admin] RBAC: Adding query filter params (name=reader) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:31:51.557397 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3f759-6163-4729-9ad3-2adc536ee3fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:51.558772 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3f759-6163-4729-9ad3-2adc536ee3fc None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:51.567331 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 55/113] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jul 6 13:31:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:51.572943 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7dc40-62d2-4eb9-a712-5de8a1c535fc None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:51.573257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7dc40-62d2-4eb9-a712-5de8a1c535fc None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:51.573546 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7dc40-62d2-4eb9-a712-5de8a1c535fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:51.593653 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45d9df-8380-436a-af63-431fd51e627b None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:51.597149 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-a1b7dc40-62d2-4eb9-a712-5de8a1c535fc None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:31:51.598340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a1b7dc40-62d2-4eb9-a712-5de8a1c535fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:31:51.631019 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e9839-899b-482e-b4b6-8688d619ff3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:51.631916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e9839-899b-482e-b4b6-8688d619ff3f None admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:51.632202 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e9839-899b-482e-b4b6-8688d619ff3f None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:51.632482 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e9839-899b-482e-b4b6-8688d619ff3f None admin] PATH_INFO: `/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93/users/c17e13a3d58f4ceab768f46d43e2cab5/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:51.634516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e9839-899b-482e-b4b6-8688d619ff3f None admin] RBAC: Authorizing `identity:create_grant(project_id=202d7ba5c7f34872aa3ab82e5d2b2f93, user_id=c17e13a3d58f4ceab768f46d43e2cab5, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:51.635964 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e9839-899b-482e-b4b6-8688d619ff3f None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:51.650763 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 59/114] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jul 6 13:31:51 2022] PUT /identity/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93/users/c17e13a3d58f4ceab768f46d43e2cab5/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:31:51.651582 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e9839-899b-482e-b4b6-8688d619ff3f None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:51.672829 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:51.693338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 56/115] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:31:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:31:51.693338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee23e3d-3ae1-4600-93b4-bd5eeb433dcf None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:51.693338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee23e3d-3ae1-4600-93b4-bd5eeb433dcf None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:51.693338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee23e3d-3ae1-4600-93b4-bd5eeb433dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:51.721656 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-aee23e3d-3ae1-4600-93b4-bd5eeb433dcf None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:31:51.723075 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-aee23e3d-3ae1-4600-93b4-bd5eeb433dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:31:51.800221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e9839-899b-482e-b4b6-8688d619ff3f None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:51.802451 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 57/116] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:31:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:31:51.809127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee23e3d-3ae1-4600-93b4-bd5eeb433dcf None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:51.820096 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:51.845273 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f893ee7-28f8-40c8-9914-d029dea0518f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:51.846228 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f893ee7-28f8-40c8-9914-d029dea0518f None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:51.846518 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f893ee7-28f8-40c8-9914-d029dea0518f None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:51.846895 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f893ee7-28f8-40c8-9914-d029dea0518f None admin] PATH_INFO: `/v3/users/31e4ee38bf8148f4a4dd9f56097514a1` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:51.848158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f893ee7-28f8-40c8-9914-d029dea0518f None admin] RBAC: Authorizing `identity:get_user(user_id=31e4ee38bf8148f4a4dd9f56097514a1)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:51.849674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f893ee7-28f8-40c8-9914-d029dea0518f None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:51.855818 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 60/117] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:31:51 2022] GET /identity/v3/users/31e4ee38bf8148f4a4dd9f56097514a1 => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:51.855818 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f893ee7-28f8-40c8-9914-d029dea0518f None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:51.927285 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee23e3d-3ae1-4600-93b4-bd5eeb433dcf None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:51.955115 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:51.965694 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ec451-96de-445f-b3bc-807e7409b78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:51.966620 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ec451-96de-445f-b3bc-807e7409b78a None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:51.966908 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ec451-96de-445f-b3bc-807e7409b78a None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:51.967203 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ec451-96de-445f-b3bc-807e7409b78a None admin] PATH_INFO: `/v3/users/33e0cd5bda0d46059c4a848ccf7a6385` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:51.974050 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f893ee7-28f8-40c8-9914-d029dea0518f None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:51.980803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ec451-96de-445f-b3bc-807e7409b78a None admin] RBAC: Authorizing `identity:get_user(user_id=33e0cd5bda0d46059c4a848ccf7a6385)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:51.982355 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ec451-96de-445f-b3bc-807e7409b78a None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:51.984078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 58/118] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:31:51 2022] GET /identity/v3/users/33e0cd5bda0d46059c4a848ccf7a6385 => generated 289 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:51.985339 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d57a48-c03b-4a7a-9b83-0847fb888e73 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:51.985627 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d57a48-c03b-4a7a-9b83-0847fb888e73 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:51.985904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d57a48-c03b-4a7a-9b83-0847fb888e73 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:51.991010 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 59/119] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:31:51.991010 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d57a48-c03b-4a7a-9b83-0847fb888e73 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:52.019429 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb84f6-4f19-4c1f-9736-7bc0dc828e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:52.020251 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb84f6-4f19-4c1f-9736-7bc0dc828e88 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:52.020539 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb84f6-4f19-4c1f-9736-7bc0dc828e88 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:52.020887 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb84f6-4f19-4c1f-9736-7bc0dc828e88 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:52.032100 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fb84f6-4f19-4c1f-9736-7bc0dc828e88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:31:52.040118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-86fb84f6-4f19-4c1f-9736-7bc0dc828e88 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 13:31:52.040118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:31:52.040118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:31:52.040118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:31:52.040118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:31:52.040118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:31:52.040118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:31:52.040118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:31:52.040118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:31:52.040118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:31:52.040118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:31:52.040118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:31:52.040118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:31:52.040118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:31:52.040118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:31:52.040118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:31:52.040948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:31:52.040948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:31:52.040948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:31:52.040948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:31:52.040948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:31:52.040948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:31:52.040948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:31:52.040948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:31:52.040948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:31:52.040948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:31:52.040948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:31:52.040948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:31:52.040948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:31:52.040948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:31:52.040948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:31:52.040948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:31:52.041991 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:31:52.041991 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:31:52.041991 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:31:52.041991 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:31:52.041991 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:31:52.041991 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:31:52.041991 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:31:52.041991 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:31:52.041991 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:31:52.041991 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:31:52.041991 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 13:31:52.041991 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:31:52.041991 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 61/120] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jul 6 13:31:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:31:52.042644 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb84f6-4f19-4c1f-9736-7bc0dc828e88 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:52.095116 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:52.097238 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d57a48-c03b-4a7a-9b83-0847fb888e73 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:52.136111 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca43678-9138-4da3-922c-eb0abd1cee52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:52.136925 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca43678-9138-4da3-922c-eb0abd1cee52 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:52.137261 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca43678-9138-4da3-922c-eb0abd1cee52 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:52.137548 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca43678-9138-4da3-922c-eb0abd1cee52 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:52.139582 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca43678-9138-4da3-922c-eb0abd1cee52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:52.141041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca43678-9138-4da3-922c-eb0abd1cee52 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:52.147028 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 60/121] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:31:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:52.147028 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca43678-9138-4da3-922c-eb0abd1cee52 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:52.161933 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb84f6-4f19-4c1f-9736-7bc0dc828e88 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:52.197723 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0285960-87e0-4946-91de-3e413c781a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:52.198640 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0285960-87e0-4946-91de-3e413c781a1d None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:52.198925 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0285960-87e0-4946-91de-3e413c781a1d None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:52.199202 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0285960-87e0-4946-91de-3e413c781a1d None admin] PATH_INFO: `/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:52.200687 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0285960-87e0-4946-91de-3e413c781a1d None admin] RBAC: Authorizing `identity:get_project(project_id=202d7ba5c7f34872aa3ab82e5d2b2f93)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:52.202404 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0285960-87e0-4946-91de-3e413c781a1d None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:52.203417 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:52.206759 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 62/122] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:31:52 2022] GET /identity/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:52.206759 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c685761-c6d6-4add-aeeb-de4cc347ae36 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:52.206950 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c685761-c6d6-4add-aeeb-de4cc347ae36 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:52.207123 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c685761-c6d6-4add-aeeb-de4cc347ae36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:52.226247 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-9c685761-c6d6-4add-aeeb-de4cc347ae36 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:31:52.227750 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-9c685761-c6d6-4add-aeeb-de4cc347ae36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:31:52.263698 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca43678-9138-4da3-922c-eb0abd1cee52 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:52.306234 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d560c418-6cfc-4617-b85d-6ea237fa1b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:52.306979 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560c418-6cfc-4617-b85d-6ea237fa1b40 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:52.307267 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560c418-6cfc-4617-b85d-6ea237fa1b40 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:52.307550 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560c418-6cfc-4617-b85d-6ea237fa1b40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:52.310848 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 63/123] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:31:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:31:52.317505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c685761-c6d6-4add-aeeb-de4cc347ae36 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:52.369119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:52.392332 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d560c418-6cfc-4617-b85d-6ea237fa1b40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:52.394402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d560c418-6cfc-4617-b85d-6ea237fa1b40 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:52.429229 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c685761-c6d6-4add-aeeb-de4cc347ae36 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:52.451812 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:52.466922 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35597100-83fd-4ec5-8fb9-e48cc0719178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:52.467087 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35597100-83fd-4ec5-8fb9-e48cc0719178 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:52.467383 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35597100-83fd-4ec5-8fb9-e48cc0719178 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:52.467606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35597100-83fd-4ec5-8fb9-e48cc0719178 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:52.468714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35597100-83fd-4ec5-8fb9-e48cc0719178 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:31:52.468714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35597100-83fd-4ec5-8fb9-e48cc0719178 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:52.470104 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35597100-83fd-4ec5-8fb9-e48cc0719178 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:52.484337 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 64/124] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:31:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:52.484337 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35597100-83fd-4ec5-8fb9-e48cc0719178 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:52.510676 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 61/125] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:31:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:31:52.511665 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d560c418-6cfc-4617-b85d-6ea237fa1b40 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:52.608640 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35597100-83fd-4ec5-8fb9-e48cc0719178 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:52.611990 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:52.618507 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d560c418-6cfc-4617-b85d-6ea237fa1b40 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:52.644735 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b73a6f-6115-4d94-bb43-1e6aecfd6c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:52.645722 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b73a6f-6115-4d94-bb43-1e6aecfd6c7d None admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:52.646239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b73a6f-6115-4d94-bb43-1e6aecfd6c7d None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:52.646637 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b73a6f-6115-4d94-bb43-1e6aecfd6c7d None admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:52.647397 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b73a6f-6115-4d94-bb43-1e6aecfd6c7d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:52.648809 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b73a6f-6115-4d94-bb43-1e6aecfd6c7d None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:52.655801 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b98af-9bdc-424f-aae1-c11f0989789f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:52.656697 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b98af-9bdc-424f-aae1-c11f0989789f None admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:52.656987 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b98af-9bdc-424f-aae1-c11f0989789f None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:52.657295 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b98af-9bdc-424f-aae1-c11f0989789f None admin] PATH_INFO: `/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93/users/31e4ee38bf8148f4a4dd9f56097514a1/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:52.659260 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b98af-9bdc-424f-aae1-c11f0989789f None admin] RBAC: Authorizing `identity:create_grant(project_id=202d7ba5c7f34872aa3ab82e5d2b2f93, user_id=31e4ee38bf8148f4a4dd9f56097514a1, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:52.660705 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b98af-9bdc-424f-aae1-c11f0989789f None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:52.677482 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 62/126] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jul 6 13:31:52 2022] PUT /identity/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93/users/31e4ee38bf8148f4a4dd9f56097514a1/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:31:52.677997 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 65/127] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Jul 6 13:31:52 2022] POST /identity/v3/projects => generated 302 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:31:52.678645 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b73a6f-6115-4d94-bb43-1e6aecfd6c7d None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:52.679265 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b98af-9bdc-424f-aae1-c11f0989789f None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:52.723932 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:52.878034 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b73a6f-6115-4d94-bb43-1e6aecfd6c7d None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:52.882354 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b98af-9bdc-424f-aae1-c11f0989789f None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:52.928552 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35cda9b-4fb2-4dcf-9c1d-79fb14aa46cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:52.928774 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35cda9b-4fb2-4dcf-9c1d-79fb14aa46cc None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:52.929122 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35cda9b-4fb2-4dcf-9c1d-79fb14aa46cc None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:52.929372 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35cda9b-4fb2-4dcf-9c1d-79fb14aa46cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:52.937793 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc348af0-d78d-4dbe-908c-60abd965ca82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:52.939231 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc348af0-d78d-4dbe-908c-60abd965ca82 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:52.939593 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc348af0-d78d-4dbe-908c-60abd965ca82 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:52.939947 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc348af0-d78d-4dbe-908c-60abd965ca82 None admin] PATH_INFO: `/v3/users/31e4ee38bf8148f4a4dd9f56097514a1` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:52.941300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc348af0-d78d-4dbe-908c-60abd965ca82 None admin] RBAC: Authorizing `identity:get_user(user_id=31e4ee38bf8148f4a4dd9f56097514a1)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:52.942809 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35cda9b-4fb2-4dcf-9c1d-79fb14aa46cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:31:52.942916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc348af0-d78d-4dbe-908c-60abd965ca82 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:52.944600 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 63/128] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:31:52 2022] GET /identity/v3/users/31e4ee38bf8148f4a4dd9f56097514a1 => generated 275 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:52.945467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc348af0-d78d-4dbe-908c-60abd965ca82 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:52.953259 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-e35cda9b-4fb2-4dcf-9c1d-79fb14aa46cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:31:52.953259 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:31:52.953259 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:31:52.953259 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:31:52.953259 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:31:52.953259 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:31:52.953259 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:31:52.953259 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:31:52.953259 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:31:52.953259 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:31:52.953259 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:31:52.953259 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:31:52.953259 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:31:52.953259 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:31:52.953259 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:31:52.953259 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:31:52.954196 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:31:52.954196 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:31:52.954196 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:31:52.954196 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:31:52.954196 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:31:52.954196 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:31:52.954196 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:31:52.954196 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:31:52.954196 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:31:52.954196 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:31:52.954196 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:31:52.954196 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:31:52.954196 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:31:52.954196 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:31:52.954196 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:31:52.954196 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:31:52.954952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:31:52.954952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:31:52.954952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:31:52.954952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:31:52.954952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:31:52.954952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:31:52.954952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:31:52.954952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:31:52.954952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:31:52.954952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:31:52.954952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:31:52.954952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:31:52.957894 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 66/129] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:31:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:31:52.958869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35cda9b-4fb2-4dcf-9c1d-79fb14aa46cc None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:52.982500 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:53.064243 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:53.086762 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35cda9b-4fb2-4dcf-9c1d-79fb14aa46cc None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:53.092946 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc348af0-d78d-4dbe-908c-60abd965ca82 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:53.124229 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062108e-4ed2-4fff-90de-a73320fbba6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:53.125522 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062108e-4ed2-4fff-90de-a73320fbba6a None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:53.125979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062108e-4ed2-4fff-90de-a73320fbba6a None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:53.126539 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062108e-4ed2-4fff-90de-a73320fbba6a None admin] PATH_INFO: `/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:53.127914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c062108e-4ed2-4fff-90de-a73320fbba6a None admin] RBAC: Authorizing `identity:get_project(project_id=202d7ba5c7f34872aa3ab82e5d2b2f93)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:53.129764 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c062108e-4ed2-4fff-90de-a73320fbba6a None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:53.131669 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 67/130] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:31:52 2022] GET /identity/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:53.132867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062108e-4ed2-4fff-90de-a73320fbba6a None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:53.143250 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0dc97e-eebe-4fc8-86d7-e3df2fd6bf6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:53.143250 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0dc97e-eebe-4fc8-86d7-e3df2fd6bf6c None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:53.143250 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0dc97e-eebe-4fc8-86d7-e3df2fd6bf6c None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:53.143250 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0dc97e-eebe-4fc8-86d7-e3df2fd6bf6c None admin] PATH_INFO: `/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:53.143618 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0dc97e-eebe-4fc8-86d7-e3df2fd6bf6c None admin] RBAC: Authorizing `identity:get_project(project_id=202d7ba5c7f34872aa3ab82e5d2b2f93)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:53.144791 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0dc97e-eebe-4fc8-86d7-e3df2fd6bf6c None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:53.145674 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 64/131] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:31:52 2022] GET /identity/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:53.146414 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0dc97e-eebe-4fc8-86d7-e3df2fd6bf6c None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:53.302098 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0dc97e-eebe-4fc8-86d7-e3df2fd6bf6c None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:53.328543 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:53.359371 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062108e-4ed2-4fff-90de-a73320fbba6a None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:53.363728 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6c24c-e84b-4f7b-9434-cecaf4672514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:53.364636 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6c24c-e84b-4f7b-9434-cecaf4672514 None admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:53.364923 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6c24c-e84b-4f7b-9434-cecaf4672514 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:53.365225 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6c24c-e84b-4f7b-9434-cecaf4672514 None admin] PATH_INFO: `/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93/users/31e4ee38bf8148f4a4dd9f56097514a1/roles/534400e7b28546a8a04c08f825c28566` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:53.372117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a6c24c-e84b-4f7b-9434-cecaf4672514 None admin] RBAC: Authorizing `identity:create_grant(project_id=202d7ba5c7f34872aa3ab82e5d2b2f93, user_id=31e4ee38bf8148f4a4dd9f56097514a1, role_id=534400e7b28546a8a04c08f825c28566)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:53.374350 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a6c24c-e84b-4f7b-9434-cecaf4672514 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:53.399930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 65/132] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jul 6 13:31:53 2022] PUT /identity/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93/users/31e4ee38bf8148f4a4dd9f56097514a1/roles/534400e7b28546a8a04c08f825c28566 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:31:53.400795 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6c24c-e84b-4f7b-9434-cecaf4672514 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:53.416229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:53.435897 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357fbdc-6248-4ee2-9b07-dcdf043745ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:53.438136 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357fbdc-6248-4ee2-9b07-dcdf043745ce None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:53.438670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357fbdc-6248-4ee2-9b07-dcdf043745ce None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:53.439180 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357fbdc-6248-4ee2-9b07-dcdf043745ce None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:53.440479 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357fbdc-6248-4ee2-9b07-dcdf043745ce None admin] RBAC: Adding query filter params (name=member) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:31:53.440993 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357fbdc-6248-4ee2-9b07-dcdf043745ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:53.442883 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357fbdc-6248-4ee2-9b07-dcdf043745ce None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:53.464446 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 68/133] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jul 6 13:31:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:53.470754 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357fbdc-6248-4ee2-9b07-dcdf043745ce None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:53.592451 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6c24c-e84b-4f7b-9434-cecaf4672514 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:53.642971 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33d79c8-d372-441e-8f8b-35baf7404333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:53.643638 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d79c8-d372-441e-8f8b-35baf7404333 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:53.643925 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d79c8-d372-441e-8f8b-35baf7404333 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:53.645044 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d79c8-d372-441e-8f8b-35baf7404333 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:53.645169 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33d79c8-d372-441e-8f8b-35baf7404333 None admin] RBAC: Adding query filter params (role.id=65ecdcc5a9d0487ba529e54c626e152f, scope.project.id=202d7ba5c7f34872aa3ab82e5d2b2f93, user.id=33e0cd5bda0d46059c4a848ccf7a6385) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:31:53.645413 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33d79c8-d372-441e-8f8b-35baf7404333 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:53.646932 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33d79c8-d372-441e-8f8b-35baf7404333 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:53.652733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 66/134] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:31:53 2022] GET /identity/v3/role_assignments?user.id=33e0cd5bda0d46059c4a848ccf7a6385&scope.project.id=202d7ba5c7f34872aa3ab82e5d2b2f93&role.id=65ecdcc5a9d0487ba529e54c626e152f => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:53.698469 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357fbdc-6248-4ee2-9b07-dcdf043745ce None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:53.754694 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f2d9c-2303-4cf7-91e7-4bb53773e324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:53.755606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f2d9c-2303-4cf7-91e7-4bb53773e324 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:53.755888 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f2d9c-2303-4cf7-91e7-4bb53773e324 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:53.756165 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f2d9c-2303-4cf7-91e7-4bb53773e324 None admin] PATH_INFO: `/v3/users/31e4ee38bf8148f4a4dd9f56097514a1` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:53.757439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f2d9c-2303-4cf7-91e7-4bb53773e324 None admin] RBAC: Authorizing `identity:get_user(user_id=31e4ee38bf8148f4a4dd9f56097514a1)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:53.758907 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f2d9c-2303-4cf7-91e7-4bb53773e324 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:53.760581 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 69/135] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:31:53 2022] GET /identity/v3/users/31e4ee38bf8148f4a4dd9f56097514a1 => generated 275 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:31:53.766492 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33d79c8-d372-441e-8f8b-35baf7404333 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:53.876282 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:53.912522 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33d79c8-d372-441e-8f8b-35baf7404333 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:53.956550 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d78674-d292-4b63-b87a-d8880615002b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:53.957475 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d78674-d292-4b63-b87a-d8880615002b None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:53.957764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d78674-d292-4b63-b87a-d8880615002b None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:53.958042 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d78674-d292-4b63-b87a-d8880615002b None admin] PATH_INFO: `/v3/projects/a3eb71934e924f9e801e61e599cba492` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:53.959248 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d78674-d292-4b63-b87a-d8880615002b None admin] RBAC: Authorizing `identity:get_project(project_id=a3eb71934e924f9e801e61e599cba492)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:53.960720 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d78674-d292-4b63-b87a-d8880615002b None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:53.962679 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 67/136] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:31:53 2022] GET /identity/v3/projects/a3eb71934e924f9e801e61e599cba492 => generated 312 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:53.969365 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f2d9c-2303-4cf7-91e7-4bb53773e324 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:53.987194 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:54.081371 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f2d9c-2303-4cf7-91e7-4bb53773e324 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:54.094283 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6b28b-2735-45ce-9705-cf6732d7c0e6 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:54.094574 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6b28b-2735-45ce-9705-cf6732d7c0e6 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:54.094860 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6b28b-2735-45ce-9705-cf6732d7c0e6 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:54.096229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 68/137] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:31:54.101513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a482ffc-0a06-4a7e-9794-50929fef6dcc None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:54.102062 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a482ffc-0a06-4a7e-9794-50929fef6dcc None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:54.102445 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a482ffc-0a06-4a7e-9794-50929fef6dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:54.125186 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-9a482ffc-0a06-4a7e-9794-50929fef6dcc None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:31:54.127202 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-9a482ffc-0a06-4a7e-9794-50929fef6dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:31:54.144310 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d9d13c-771c-4050-be3e-3e37c59a0c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:54.145249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9d13c-771c-4050-be3e-3e37c59a0c88 None admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:54.145540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9d13c-771c-4050-be3e-3e37c59a0c88 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:54.145821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9d13c-771c-4050-be3e-3e37c59a0c88 None admin] PATH_INFO: `/v3/projects/a3eb71934e924f9e801e61e599cba492/users/31e4ee38bf8148f4a4dd9f56097514a1/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:54.150338 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d9d13c-771c-4050-be3e-3e37c59a0c88 None admin] RBAC: Authorizing `identity:create_grant(project_id=a3eb71934e924f9e801e61e599cba492, user_id=31e4ee38bf8148f4a4dd9f56097514a1, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:54.151798 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d9d13c-771c-4050-be3e-3e37c59a0c88 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:54.179633 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 70/138] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jul 6 13:31:53 2022] PUT /identity/v3/projects/a3eb71934e924f9e801e61e599cba492/users/31e4ee38bf8148f4a4dd9f56097514a1/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:31:54.209207 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 69/139] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:31:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 13:31:54.238908 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d9d13c-771c-4050-be3e-3e37c59a0c88 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:54.322949 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:54.352524 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d9d13c-771c-4050-be3e-3e37c59a0c88 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:54.393536 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db17ff-d9ce-4d8a-96c1-3f5b44c4995a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:54.394280 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db17ff-d9ce-4d8a-96c1-3f5b44c4995a None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:54.394566 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db17ff-d9ce-4d8a-96c1-3f5b44c4995a None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:54.394906 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db17ff-d9ce-4d8a-96c1-3f5b44c4995a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:54.414572 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db17ff-d9ce-4d8a-96c1-3f5b44c4995a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:31:54.423639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-f1db17ff-d9ce-4d8a-96c1-3f5b44c4995a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:31:54.423639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:31:54.423639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:31:54.423639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:31:54.423639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:31:54.423639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:31:54.423639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:31:54.423639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:31:54.423639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:31:54.423639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:31:54.423639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:31:54.423639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:31:54.423639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:31:54.423639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:31:54.423639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:31:54.423639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:31:54.424810 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:31:54.424810 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:31:54.424810 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:31:54.424810 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:31:54.424810 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:31:54.424810 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:31:54.424810 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:31:54.424810 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:31:54.424810 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:31:54.424810 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:31:54.424810 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:31:54.424810 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:31:54.424810 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:31:54.424810 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:31:54.424810 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:31:54.424810 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:31:54.425596 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:31:54.425596 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:31:54.425596 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:31:54.425596 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:31:54.425596 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:31:54.425596 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:31:54.425596 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:31:54.425596 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:31:54.425596 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:31:54.425596 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:31:54.425596 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:31:54.425596 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:31:54.425596 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 71/140] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:31:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:31:54.431243 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a482ffc-0a06-4a7e-9794-50929fef6dcc None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:54.506275 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:54.541056 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a482ffc-0a06-4a7e-9794-50929fef6dcc None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:54.577658 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d832f-b20f-41fa-90aa-53837682d713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:54.578344 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d832f-b20f-41fa-90aa-53837682d713 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:54.578632 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d832f-b20f-41fa-90aa-53837682d713 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:54.578912 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d832f-b20f-41fa-90aa-53837682d713 None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:54.579676 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4d832f-b20f-41fa-90aa-53837682d713 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:31:54.579988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4d832f-b20f-41fa-90aa-53837682d713 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:54.581424 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4d832f-b20f-41fa-90aa-53837682d713 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:54.590880 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 70/141] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:31:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:54.598497 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db17ff-d9ce-4d8a-96c1-3f5b44c4995a None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:54.705517 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db17ff-d9ce-4d8a-96c1-3f5b44c4995a None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:54.740237 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf31c11-f515-46d2-a9ee-9d1a7bf9b14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:54.741148 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf31c11-f515-46d2-a9ee-9d1a7bf9b14c None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:54.741441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf31c11-f515-46d2-a9ee-9d1a7bf9b14c None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:54.741732 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf31c11-f515-46d2-a9ee-9d1a7bf9b14c None admin] PATH_INFO: `/v3/users/c17e13a3d58f4ceab768f46d43e2cab5` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:54.742953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf31c11-f515-46d2-a9ee-9d1a7bf9b14c None admin] RBAC: Authorizing `identity:get_user(user_id=c17e13a3d58f4ceab768f46d43e2cab5)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:54.744449 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf31c11-f515-46d2-a9ee-9d1a7bf9b14c None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:54.746119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 72/142] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:31:54 2022] GET /identity/v3/users/c17e13a3d58f4ceab768f46d43e2cab5 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:54.752686 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d832f-b20f-41fa-90aa-53837682d713 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:54.862133 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d832f-b20f-41fa-90aa-53837682d713 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:54.893485 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:54.902589 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6414b1-9a3c-4f9d-8fa1-5a5bd1b265af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:54.903286 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6414b1-9a3c-4f9d-8fa1-5a5bd1b265af None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:54.903580 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6414b1-9a3c-4f9d-8fa1-5a5bd1b265af None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:54.905203 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6414b1-9a3c-4f9d-8fa1-5a5bd1b265af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:55.000492 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6414b1-9a3c-4f9d-8fa1-5a5bd1b265af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:55.001993 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6414b1-9a3c-4f9d-8fa1-5a5bd1b265af None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:55.133082 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 71/143] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:31:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:31:55.139307 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf31c11-f515-46d2-a9ee-9d1a7bf9b14c None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:55.142213 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:55.246208 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf31c11-f515-46d2-a9ee-9d1a7bf9b14c None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:55.284597 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb227e-c817-41bf-83e3-28a15379c204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:55.285434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb227e-c817-41bf-83e3-28a15379c204 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:55.285716 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb227e-c817-41bf-83e3-28a15379c204 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:55.285992 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb227e-c817-41bf-83e3-28a15379c204 None admin] PATH_INFO: `/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:55.287284 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb227e-c817-41bf-83e3-28a15379c204 None admin] RBAC: Authorizing `identity:get_project(project_id=202d7ba5c7f34872aa3ab82e5d2b2f93)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:55.288773 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb227e-c817-41bf-83e3-28a15379c204 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:55.290956 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 73/144] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:31:55 2022] GET /identity/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:55.297630 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6414b1-9a3c-4f9d-8fa1-5a5bd1b265af None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:55.332436 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:55.400421 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6414b1-9a3c-4f9d-8fa1-5a5bd1b265af None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:55.435872 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca3718-18c2-41a1-8fcf-c188cde1ff5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:55.436596 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca3718-18c2-41a1-8fcf-c188cde1ff5c None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:55.436876 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca3718-18c2-41a1-8fcf-c188cde1ff5c None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:55.437193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca3718-18c2-41a1-8fcf-c188cde1ff5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:55.438031 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca3718-18c2-41a1-8fcf-c188cde1ff5c None admin] RBAC: Adding query filter params (role.id=795daa2c78354fd48e4a741e1182cc98, scope.project.id=202d7ba5c7f34872aa3ab82e5d2b2f93, user.id=c17e13a3d58f4ceab768f46d43e2cab5) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:31:55.438342 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca3718-18c2-41a1-8fcf-c188cde1ff5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:55.439777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca3718-18c2-41a1-8fcf-c188cde1ff5c None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:55.446278 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 72/145] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:31:55 2022] GET /identity/v3/role_assignments?user.id=c17e13a3d58f4ceab768f46d43e2cab5&scope.project.id=202d7ba5c7f34872aa3ab82e5d2b2f93&role.id=795daa2c78354fd48e4a741e1182cc98 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:55.476221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ebc7c-3cc3-4246-aaa6-e0947d7621ca None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:55.476743 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ebc7c-3cc3-4246-aaa6-e0947d7621ca None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:55.476817 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ebc7c-3cc3-4246-aaa6-e0947d7621ca None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:55.478538 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 74/146] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:31:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:31:55.484002 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58d4c9-13e1-4800-b644-e235e5486998 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:55.484292 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58d4c9-13e1-4800-b644-e235e5486998 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:55.484556 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58d4c9-13e1-4800-b644-e235e5486998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:55.506187 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-4d58d4c9-13e1-4800-b644-e235e5486998 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:31:55.507395 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-4d58d4c9-13e1-4800-b644-e235e5486998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:31:55.588578 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 73/147] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:31:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:31:55.611424 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377ebc7c-3cc3-4246-aaa6-e0947d7621ca None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:55.688748 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:55.723073 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377ebc7c-3cc3-4246-aaa6-e0947d7621ca None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:55.759705 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4647b1-848f-47a7-9219-20db8ecd3c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:55.760504 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4647b1-848f-47a7-9219-20db8ecd3c0c None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:55.760898 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4647b1-848f-47a7-9219-20db8ecd3c0c None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:55.761298 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4647b1-848f-47a7-9219-20db8ecd3c0c None admin] PATH_INFO: `/v3/roles/member` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:55.773484 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4647b1-848f-47a7-9219-20db8ecd3c0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:31:55.781916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-fd4647b1-848f-47a7-9219-20db8ecd3c0c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:31:55.781916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:31:55.781916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:31:55.781916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:31:55.781916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:31:55.781916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:31:55.781916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:31:55.781916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:31:55.781916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:31:55.781916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:31:55.781916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:31:55.781916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:31:55.781916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:31:55.781916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:31:55.781916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:31:55.781916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:31:55.782830 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:31:55.782830 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:31:55.782830 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:31:55.782830 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:31:55.782830 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:31:55.782830 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:31:55.782830 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:31:55.782830 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:31:55.782830 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:31:55.782830 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:31:55.782830 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:31:55.782830 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:31:55.782830 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:31:55.782830 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:31:55.782830 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:31:55.782830 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:31:55.783584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:31:55.783584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:31:55.783584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:31:55.783584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:31:55.783584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:31:55.783584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:31:55.783584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:31:55.783584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:31:55.783584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:31:55.783584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:31:55.783584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:31:55.783584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:31:55.783584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 75/148] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:31:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:31:55.789149 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58d4c9-13e1-4800-b644-e235e5486998 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:55.817349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:55.895940 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58d4c9-13e1-4800-b644-e235e5486998 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:55.931822 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae7dda-497f-4525-8f01-936f961a071c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:55.932495 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae7dda-497f-4525-8f01-936f961a071c None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:55.932778 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae7dda-497f-4525-8f01-936f961a071c None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:55.933086 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae7dda-497f-4525-8f01-936f961a071c None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:55.933842 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ae7dda-497f-4525-8f01-936f961a071c None admin] RBAC: Adding query filter params (name=member) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:31:55.934150 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ae7dda-497f-4525-8f01-936f961a071c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:55.935854 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ae7dda-497f-4525-8f01-936f961a071c None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:55.944677 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 74/149] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jul 6 13:31:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:55.950127 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4647b1-848f-47a7-9219-20db8ecd3c0c None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:56.054538 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4647b1-848f-47a7-9219-20db8ecd3c0c None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:56.083898 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:56.090135 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a026f-0f85-43dc-84ff-064af158e36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:56.091411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a026f-0f85-43dc-84ff-064af158e36f None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:56.092230 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a026f-0f85-43dc-84ff-064af158e36f None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:56.092985 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a026f-0f85-43dc-84ff-064af158e36f None admin] PATH_INFO: `/v3/users/31e4ee38bf8148f4a4dd9f56097514a1` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:56.098359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929a026f-0f85-43dc-84ff-064af158e36f None admin] RBAC: Authorizing `identity:get_user(user_id=31e4ee38bf8148f4a4dd9f56097514a1)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:56.099805 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929a026f-0f85-43dc-84ff-064af158e36f None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:56.101484 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 76/150] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:31:55 2022] GET /identity/v3/users/31e4ee38bf8148f4a4dd9f56097514a1 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:56.107665 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae7dda-497f-4525-8f01-936f961a071c None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:56.147544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c1ad0-fa7e-4f4d-a7b4-308f7aabcb1a None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:56.148122 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c1ad0-fa7e-4f4d-a7b4-308f7aabcb1a None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:56.148590 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c1ad0-fa7e-4f4d-a7b4-308f7aabcb1a None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:56.150242 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 77/151] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:31:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:31:56.155435 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55d062-9554-4c05-9caf-2b581b0a8483 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:56.155988 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55d062-9554-4c05-9caf-2b581b0a8483 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:56.156458 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55d062-9554-4c05-9caf-2b581b0a8483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:56.175251 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-ef55d062-9554-4c05-9caf-2b581b0a8483 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:31:56.176764 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-ef55d062-9554-4c05-9caf-2b581b0a8483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:31:56.210151 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:56.227660 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae7dda-497f-4525-8f01-936f961a071c None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:56.272014 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 78/152] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:31:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 13:31:56.273519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36f4df-9ae9-4a60-af64-6957d11696ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:56.274174 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36f4df-9ae9-4a60-af64-6957d11696ca None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:56.274457 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36f4df-9ae9-4a60-af64-6957d11696ca None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:56.274731 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36f4df-9ae9-4a60-af64-6957d11696ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:56.314115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55d062-9554-4c05-9caf-2b581b0a8483 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:56.416731 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36f4df-9ae9-4a60-af64-6957d11696ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:56.418239 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36f4df-9ae9-4a60-af64-6957d11696ca None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:56.431401 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55d062-9554-4c05-9caf-2b581b0a8483 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:56.459541 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:56.469055 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a7998a-9f14-4b9c-a603-93a2da74271a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:56.469953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7998a-9f14-4b9c-a603-93a2da74271a None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:56.470299 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7998a-9f14-4b9c-a603-93a2da74271a None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:56.470642 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7998a-9f14-4b9c-a603-93a2da74271a None admin] PATH_INFO: `/v3/domains/default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:56.471880 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a7998a-9f14-4b9c-a603-93a2da74271a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:56.473423 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a7998a-9f14-4b9c-a603-93a2da74271a None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:56.475444 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 79/153] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:31:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:31:56.476815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05684ea9-c4d5-42c8-8d1a-ca91d4a042eb None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:56.477363 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05684ea9-c4d5-42c8-8d1a-ca91d4a042eb None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:56.477451 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05684ea9-c4d5-42c8-8d1a-ca91d4a042eb None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:56.478714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 80/154] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:31:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:31:56.484105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05684ea9-c4d5-42c8-8d1a-ca91d4a042eb None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:56.517454 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:56.532767 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 75/155] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:31:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:31:56.534425 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba292f-76b3-4a23-85d5-f3885452377c None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:56.534928 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba292f-76b3-4a23-85d5-f3885452377c None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:56.535309 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba292f-76b3-4a23-85d5-f3885452377c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:56.553314 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-32ba292f-76b3-4a23-85d5-f3885452377c None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:31:56.554720 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-32ba292f-76b3-4a23-85d5-f3885452377c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:31:56.608245 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05684ea9-c4d5-42c8-8d1a-ca91d4a042eb None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:56.629807 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 76/156] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:31:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:31:56.630662 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba292f-76b3-4a23-85d5-f3885452377c None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:56.661335 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a0d00-89a6-42b9-8e87-aaf569d4d71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:56.662202 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a0d00-89a6-42b9-8e87-aaf569d4d71b None admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:56.662672 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a0d00-89a6-42b9-8e87-aaf569d4d71b None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:56.663115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a0d00-89a6-42b9-8e87-aaf569d4d71b None admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:56.664049 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a0d00-89a6-42b9-8e87-aaf569d4d71b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:56.665626 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a0d00-89a6-42b9-8e87-aaf569d4d71b None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:56.697154 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 81/157] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jul 6 13:31:56 2022] POST /identity/v3/users => generated 283 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:31:56.698166 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a0d00-89a6-42b9-8e87-aaf569d4d71b None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:56.751300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba292f-76b3-4a23-85d5-f3885452377c None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:56.768953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:56.770415 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:56.792184 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c098f05-cef7-4ac3-a697-06e8be2acc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:56.793090 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c098f05-cef7-4ac3-a697-06e8be2acc0f None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:56.793381 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c098f05-cef7-4ac3-a697-06e8be2acc0f None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:56.793656 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c098f05-cef7-4ac3-a697-06e8be2acc0f None admin] PATH_INFO: `/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:56.795532 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c098f05-cef7-4ac3-a697-06e8be2acc0f None admin] RBAC: Authorizing `identity:get_project(project_id=202d7ba5c7f34872aa3ab82e5d2b2f93)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:56.796993 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c098f05-cef7-4ac3-a697-06e8be2acc0f None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:56.798821 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 77/158] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:31:56 2022] GET /identity/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:56.806234 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c098f05-cef7-4ac3-a697-06e8be2acc0f None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:56.826994 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a0d00-89a6-42b9-8e87-aaf569d4d71b None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:56.867153 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea3d849-ed0a-4c33-8ebb-55a672c71931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:56.868050 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3d849-ed0a-4c33-8ebb-55a672c71931 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:56.868338 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3d849-ed0a-4c33-8ebb-55a672c71931 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:56.868613 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3d849-ed0a-4c33-8ebb-55a672c71931 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:56.881771 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea3d849-ed0a-4c33-8ebb-55a672c71931 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:31:56.891865 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-9ea3d849-ed0a-4c33-8ebb-55a672c71931 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 13:31:56.891865 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:31:56.891865 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:31:56.891865 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:31:56.891865 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:31:56.891865 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:31:56.891865 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:31:56.891865 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:31:56.891865 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:31:56.891865 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:31:56.891865 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:31:56.891865 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:31:56.891865 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:31:56.891865 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:31:56.891865 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:31:56.891865 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:31:56.894358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:31:56.894358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:31:56.894358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:31:56.894358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:31:56.894358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:31:56.894358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:31:56.894358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:31:56.894358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:31:56.894358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:31:56.894358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:31:56.894358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:31:56.894358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:31:56.894358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:31:56.894358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:31:56.894358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:31:56.894358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:31:56.896299 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:31:56.896299 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:31:56.896299 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:31:56.896299 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:31:56.896299 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:31:56.896299 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:31:56.896299 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:31:56.896299 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:31:56.896299 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:31:56.896299 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:31:56.896299 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 13:31:56.896299 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:31:56.896299 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 82/159] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jul 6 13:31:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:31:56.906238 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea3d849-ed0a-4c33-8ebb-55a672c71931 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:56.983712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c098f05-cef7-4ac3-a697-06e8be2acc0f None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:57.032374 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3b8a7d-7625-49b2-b1ba-314ceb55f6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:57.033174 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b8a7d-7625-49b2-b1ba-314ceb55f6f2 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:57.033463 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b8a7d-7625-49b2-b1ba-314ceb55f6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:57.033743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b8a7d-7625-49b2-b1ba-314ceb55f6f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:57.034591 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3b8a7d-7625-49b2-b1ba-314ceb55f6f2 None admin] RBAC: Adding query filter params (role.id=b3244d1d9f004c1b8d3594fd30da1024, scope.project.id=202d7ba5c7f34872aa3ab82e5d2b2f93, user.id=31e4ee38bf8148f4a4dd9f56097514a1) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:31:57.034904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3b8a7d-7625-49b2-b1ba-314ceb55f6f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:57.036567 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3b8a7d-7625-49b2-b1ba-314ceb55f6f2 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:57.039041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea3d849-ed0a-4c33-8ebb-55a672c71931 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:57.045022 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 78/160] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:31:56 2022] GET /identity/v3/role_assignments?user.id=31e4ee38bf8148f4a4dd9f56097514a1&scope.project.id=202d7ba5c7f34872aa3ab82e5d2b2f93&role.id=b3244d1d9f004c1b8d3594fd30da1024 => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:57.065375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:57.082445 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd8a0b-41df-4da1-9ad5-faae70113ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:57.083129 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd8a0b-41df-4da1-9ad5-faae70113ed6 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:57.083410 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd8a0b-41df-4da1-9ad5-faae70113ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:57.083686 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd8a0b-41df-4da1-9ad5-faae70113ed6 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:57.084451 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd8a0b-41df-4da1-9ad5-faae70113ed6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:31:57.084758 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd8a0b-41df-4da1-9ad5-faae70113ed6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:57.086180 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd8a0b-41df-4da1-9ad5-faae70113ed6 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:57.096835 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 83/161] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:31:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:31:57.102334 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3b8a7d-7625-49b2-b1ba-314ceb55f6f2 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:57.114110 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a869bc-0fe4-4ee3-8628-2fc132c5ffbf None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:57.114769 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a869bc-0fe4-4ee3-8628-2fc132c5ffbf None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:57.115273 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a869bc-0fe4-4ee3-8628-2fc132c5ffbf None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:57.116948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 84/162] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:31:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:31:57.123771 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c338688-b90c-42d7-b118-f04e55e4da24 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:57.124424 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c338688-b90c-42d7-b118-f04e55e4da24 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:57.124904 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c338688-b90c-42d7-b118-f04e55e4da24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:57.146142 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-5c338688-b90c-42d7-b118-f04e55e4da24 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:31:57.147418 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-5c338688-b90c-42d7-b118-f04e55e4da24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:31:57.199077 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:57.251674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 85/163] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:31:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 13:31:57.258819 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3b8a7d-7625-49b2-b1ba-314ceb55f6f2 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:57.276269 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c338688-b90c-42d7-b118-f04e55e4da24 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:57.320773 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737d16c-e2d5-4b1e-81a9-d07734b90b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:57.321990 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737d16c-e2d5-4b1e-81a9-d07734b90b4e None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:57.322447 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737d16c-e2d5-4b1e-81a9-d07734b90b4e None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:57.322891 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737d16c-e2d5-4b1e-81a9-d07734b90b4e None admin] PATH_INFO: `/v3/users/31e4ee38bf8148f4a4dd9f56097514a1` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:57.325144 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737d16c-e2d5-4b1e-81a9-d07734b90b4e None admin] RBAC: Authorizing `identity:get_user(user_id=31e4ee38bf8148f4a4dd9f56097514a1)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:57.327504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737d16c-e2d5-4b1e-81a9-d07734b90b4e None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:57.329387 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 79/164] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:31:57 2022] GET /identity/v3/users/31e4ee38bf8148f4a4dd9f56097514a1 => generated 275 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:57.335708 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737d16c-e2d5-4b1e-81a9-d07734b90b4e None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:57.417657 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c338688-b90c-42d7-b118-f04e55e4da24 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:57.421813 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:57.459784 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054175a-06ff-4e54-83bf-40c72f3871f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:57.460905 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054175a-06ff-4e54-83bf-40c72f3871f6 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:57.461401 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054175a-06ff-4e54-83bf-40c72f3871f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:57.461926 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054175a-06ff-4e54-83bf-40c72f3871f6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:57.476633 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6054175a-06ff-4e54-83bf-40c72f3871f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:31:57.480569 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737d16c-e2d5-4b1e-81a9-d07734b90b4e None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:57.485953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-6054175a-06ff-4e54-83bf-40c72f3871f6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 13:31:57.485953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:31:57.485953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:31:57.485953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:31:57.485953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:31:57.485953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:31:57.485953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:31:57.485953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:31:57.485953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:31:57.485953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:31:57.485953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:31:57.485953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:31:57.485953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:31:57.485953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:31:57.485953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:31:57.485953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:31:57.487222 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:31:57.487222 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:31:57.487222 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:31:57.487222 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:31:57.487222 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:31:57.487222 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:31:57.487222 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:31:57.487222 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:31:57.487222 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:31:57.487222 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:31:57.487222 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:31:57.487222 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:31:57.487222 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:31:57.487222 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:31:57.487222 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:31:57.487222 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:31:57.487984 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:31:57.487984 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:31:57.487984 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:31:57.487984 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:31:57.487984 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:31:57.487984 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:31:57.487984 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:31:57.487984 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:31:57.487984 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:31:57.487984 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:31:57.487984 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 13:31:57.487984 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:31:57.495558 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 86/165] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:31:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:31:57.497095 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054175a-06ff-4e54-83bf-40c72f3871f6 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:57.507915 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:57.526484 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8570582-6936-426c-a19a-ed382486edde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:57.527399 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8570582-6936-426c-a19a-ed382486edde None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:57.527714 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8570582-6936-426c-a19a-ed382486edde None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:57.527944 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8570582-6936-426c-a19a-ed382486edde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:57.628808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8570582-6936-426c-a19a-ed382486edde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:57.630648 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8570582-6936-426c-a19a-ed382486edde None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:57.674987 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054175a-06ff-4e54-83bf-40c72f3871f6 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:57.729667 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad0de4-0a93-4806-961a-95c0763a4090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:57.730798 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad0de4-0a93-4806-961a-95c0763a4090 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:57.731080 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad0de4-0a93-4806-961a-95c0763a4090 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:57.731354 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad0de4-0a93-4806-961a-95c0763a4090 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:57.733497 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ad0de4-0a93-4806-961a-95c0763a4090 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:31:57.733822 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ad0de4-0a93-4806-961a-95c0763a4090 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:57.735470 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ad0de4-0a93-4806-961a-95c0763a4090 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:57.746671 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 87/166] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jul 6 13:31:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:31:57.749477 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcdc029-9dd4-468c-99fc-7a6af68bb0a8 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:57.749768 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcdc029-9dd4-468c-99fc-7a6af68bb0a8 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:57.750045 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcdc029-9dd4-468c-99fc-7a6af68bb0a8 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:57.752018 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 88/167] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:31:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:31:57.755287 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcdc029-9dd4-468c-99fc-7a6af68bb0a8 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:57.759336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 80/168] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:31:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:31:57.768561 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219789f7-2381-43b5-8b35-0f3a0bd3bfde None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:57.768879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219789f7-2381-43b5-8b35-0f3a0bd3bfde None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:57.769169 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219789f7-2381-43b5-8b35-0f3a0bd3bfde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:57.788907 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:57.791305 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-219789f7-2381-43b5-8b35-0f3a0bd3bfde None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:31:57.792495 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-219789f7-2381-43b5-8b35-0f3a0bd3bfde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:31:57.811958 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:57.871938 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 81/169] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:31:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 13:31:57.873309 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219789f7-2381-43b5-8b35-0f3a0bd3bfde None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:57.962630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcdc029-9dd4-468c-99fc-7a6af68bb0a8 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:58.015040 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219789f7-2381-43b5-8b35-0f3a0bd3bfde None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:58.051905 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e49e67-20a7-4237-a619-9393ca3ad174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:58.053419 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e49e67-20a7-4237-a619-9393ca3ad174 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:58.054619 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e49e67-20a7-4237-a619-9393ca3ad174 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:58.055265 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e49e67-20a7-4237-a619-9393ca3ad174 None admin] PATH_INFO: `/v3/users/33e0cd5bda0d46059c4a848ccf7a6385` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:58.058635 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e49e67-20a7-4237-a619-9393ca3ad174 None admin] RBAC: Authorizing `identity:get_user(user_id=33e0cd5bda0d46059c4a848ccf7a6385)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:58.061500 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e49e67-20a7-4237-a619-9393ca3ad174 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:58.063582 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 89/170] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:31:57 2022] GET /identity/v3/users/33e0cd5bda0d46059c4a848ccf7a6385 => generated 289 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:58.067779 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed4023-bba1-4a4b-a460-51b7bde19731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:58.069744 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e49e67-20a7-4237-a619-9393ca3ad174 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:58.070093 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed4023-bba1-4a4b-a460-51b7bde19731 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:58.070569 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed4023-bba1-4a4b-a460-51b7bde19731 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:58.071063 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed4023-bba1-4a4b-a460-51b7bde19731 None admin] PATH_INFO: `/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:58.073213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ed4023-bba1-4a4b-a460-51b7bde19731 None admin] RBAC: Authorizing `identity:get_project(project_id=202d7ba5c7f34872aa3ab82e5d2b2f93)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:58.077636 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ed4023-bba1-4a4b-a460-51b7bde19731 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:58.082169 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 82/171] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:31:57 2022] GET /identity/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:58.083401 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed4023-bba1-4a4b-a460-51b7bde19731 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:58.117784 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:58.247538 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:58.247732 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed4023-bba1-4a4b-a460-51b7bde19731 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:58.255020 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e49e67-20a7-4237-a619-9393ca3ad174 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:58.297314 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b8951-a461-40cd-b750-9dec5893789c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:58.298454 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b8951-a461-40cd-b750-9dec5893789c None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:58.299183 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b8951-a461-40cd-b750-9dec5893789c None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:58.299669 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b8951-a461-40cd-b750-9dec5893789c None admin] PATH_INFO: `/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:58.301136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b8951-a461-40cd-b750-9dec5893789c None admin] RBAC: Authorizing `identity:get_project(project_id=202d7ba5c7f34872aa3ab82e5d2b2f93)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:58.302784 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b8951-a461-40cd-b750-9dec5893789c None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:58.304976 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 83/172] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:31:58 2022] GET /identity/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:58.305982 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b8951-a461-40cd-b750-9dec5893789c None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:58.345137 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07894b3-8e96-43fb-ac87-c5a74dcb4a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:58.345947 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07894b3-8e96-43fb-ac87-c5a74dcb4a46 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:58.346233 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07894b3-8e96-43fb-ac87-c5a74dcb4a46 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:58.346516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07894b3-8e96-43fb-ac87-c5a74dcb4a46 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:58.378498 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07894b3-8e96-43fb-ac87-c5a74dcb4a46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:31:58.393923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-d07894b3-8e96-43fb-ac87-c5a74dcb4a46 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:31:58.393923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:31:58.393923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:31:58.393923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:31:58.393923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:31:58.393923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:31:58.393923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:31:58.393923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:31:58.393923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:31:58.393923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:31:58.393923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:31:58.393923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:31:58.393923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:31:58.393923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:31:58.393923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:31:58.393923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:31:58.394900 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:31:58.394900 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:31:58.394900 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:31:58.394900 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:31:58.394900 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:31:58.394900 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:31:58.394900 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:31:58.394900 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:31:58.394900 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:31:58.394900 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:31:58.394900 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:31:58.394900 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:31:58.394900 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:31:58.394900 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:31:58.394900 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:31:58.394900 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:31:58.395676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:31:58.395676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:31:58.395676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:31:58.395676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:31:58.395676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:31:58.395676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:31:58.395676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:31:58.395676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:31:58.395676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:31:58.395676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:31:58.395676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:31:58.395676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:31:58.395676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 90/173] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:31:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:31:58.396323 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07894b3-8e96-43fb-ac87-c5a74dcb4a46 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:58.397016 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:58.412837 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b8951-a461-40cd-b750-9dec5893789c None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:58.449407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b043e3-267e-466d-bd61-6c0837c02a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:58.450201 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b043e3-267e-466d-bd61-6c0837c02a77 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:58.450610 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b043e3-267e-466d-bd61-6c0837c02a77 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:58.451005 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b043e3-267e-466d-bd61-6c0837c02a77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:58.452086 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b043e3-267e-466d-bd61-6c0837c02a77 None admin] RBAC: Adding query filter params (role.id=534400e7b28546a8a04c08f825c28566, scope.project.id=202d7ba5c7f34872aa3ab82e5d2b2f93, user.id=31e4ee38bf8148f4a4dd9f56097514a1) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:31:58.452498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b043e3-267e-466d-bd61-6c0837c02a77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:58.454053 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b043e3-267e-466d-bd61-6c0837c02a77 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:58.460207 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 84/174] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:31:58 2022] GET /identity/v3/role_assignments?user.id=31e4ee38bf8148f4a4dd9f56097514a1&scope.project.id=202d7ba5c7f34872aa3ab82e5d2b2f93&role.id=534400e7b28546a8a04c08f825c28566 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:58.461222 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b043e3-267e-466d-bd61-6c0837c02a77 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:58.570593 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b043e3-267e-466d-bd61-6c0837c02a77 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:58.609984 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12460dcb-a33a-411f-92dc-d9d206f9b02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:58.611088 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12460dcb-a33a-411f-92dc-d9d206f9b02b None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:58.611469 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12460dcb-a33a-411f-92dc-d9d206f9b02b None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:58.611844 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12460dcb-a33a-411f-92dc-d9d206f9b02b None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:58.613196 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12460dcb-a33a-411f-92dc-d9d206f9b02b None admin] RBAC: Adding query filter params (name=member) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:31:58.613631 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12460dcb-a33a-411f-92dc-d9d206f9b02b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:58.615126 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12460dcb-a33a-411f-92dc-d9d206f9b02b None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:58.625727 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 85/175] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jul 6 13:31:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:58.631402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12460dcb-a33a-411f-92dc-d9d206f9b02b None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:58.675306 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07894b3-8e96-43fb-ac87-c5a74dcb4a46 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:58.681403 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:58.757569 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12460dcb-a33a-411f-92dc-d9d206f9b02b None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:58.786023 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35975a75-d26c-448d-bdd2-d2877ec70f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:58.786931 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35975a75-d26c-448d-bdd2-d2877ec70f29 None admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:58.787220 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35975a75-d26c-448d-bdd2-d2877ec70f29 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:58.787500 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35975a75-d26c-448d-bdd2-d2877ec70f29 None admin] PATH_INFO: `/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93/users/33e0cd5bda0d46059c4a848ccf7a6385/roles/65ecdcc5a9d0487ba529e54c626e152f` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:58.790855 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35975a75-d26c-448d-bdd2-d2877ec70f29 None admin] RBAC: Authorizing `identity:create_grant(project_id=202d7ba5c7f34872aa3ab82e5d2b2f93, user_id=33e0cd5bda0d46059c4a848ccf7a6385, role_id=65ecdcc5a9d0487ba529e54c626e152f)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:58.792297 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35975a75-d26c-448d-bdd2-d2877ec70f29 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:58.794120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b575973-2566-48c2-91fe-aacc644a4d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:58.795929 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b575973-2566-48c2-91fe-aacc644a4d2d None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:58.796312 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b575973-2566-48c2-91fe-aacc644a4d2d None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:58.796683 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b575973-2566-48c2-91fe-aacc644a4d2d None admin] PATH_INFO: `/v3/users/31e4ee38bf8148f4a4dd9f56097514a1` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:58.798053 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b575973-2566-48c2-91fe-aacc644a4d2d None admin] RBAC: Authorizing `identity:get_user(user_id=31e4ee38bf8148f4a4dd9f56097514a1)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:58.799844 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b575973-2566-48c2-91fe-aacc644a4d2d None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:58.802281 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 86/176] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:31:58 2022] GET /identity/v3/users/31e4ee38bf8148f4a4dd9f56097514a1 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:31:58.808059 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b575973-2566-48c2-91fe-aacc644a4d2d None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:58.827591 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 91/177] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jul 6 13:31:58 2022] PUT /identity/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93/users/33e0cd5bda0d46059c4a848ccf7a6385/roles/65ecdcc5a9d0487ba529e54c626e152f => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:31:58.895905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:58.918594 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b575973-2566-48c2-91fe-aacc644a4d2d None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:58.960855 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89378d7a-edd3-4a03-bbb8-806e47bd969e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:58.961926 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89378d7a-edd3-4a03-bbb8-806e47bd969e None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:58.962408 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89378d7a-edd3-4a03-bbb8-806e47bd969e None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:58.962880 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89378d7a-edd3-4a03-bbb8-806e47bd969e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:59.047539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89378d7a-edd3-4a03-bbb8-806e47bd969e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:59.049313 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89378d7a-edd3-4a03-bbb8-806e47bd969e None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:59.133629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:59.157547 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 87/178] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:31:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 06 13:31:59.163017 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35975a75-d26c-448d-bdd2-d2877ec70f29 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:59.182042 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:59.272096 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35975a75-d26c-448d-bdd2-d2877ec70f29 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:59.318475 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b49d1-8a18-48f1-9999-cf586a895038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:59.319342 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b49d1-8a18-48f1-9999-cf586a895038 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:59.319626 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b49d1-8a18-48f1-9999-cf586a895038 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:59.319901 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b49d1-8a18-48f1-9999-cf586a895038 None admin] PATH_INFO: `/v3/projects/a3eb71934e924f9e801e61e599cba492` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:59.321131 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82b49d1-8a18-48f1-9999-cf586a895038 None admin] RBAC: Authorizing `identity:get_project(project_id=a3eb71934e924f9e801e61e599cba492)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:59.322575 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82b49d1-8a18-48f1-9999-cf586a895038 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:59.324455 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 92/179] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:31:59 2022] GET /identity/v3/projects/a3eb71934e924f9e801e61e599cba492 => generated 312 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:59.331047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89378d7a-edd3-4a03-bbb8-806e47bd969e None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:31:59.476797 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89378d7a-edd3-4a03-bbb8-806e47bd969e None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:31:59.511386 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe543f-4cd4-4c2b-8df9-5a2e6b485f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:31:59.512426 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe543f-4cd4-4c2b-8df9-5a2e6b485f84 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:59.513226 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe543f-4cd4-4c2b-8df9-5a2e6b485f84 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:59.513919 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe543f-4cd4-4c2b-8df9-5a2e6b485f84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:59.514961 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe543f-4cd4-4c2b-8df9-5a2e6b485f84 None admin] RBAC: Adding query filter params (role.id=b3244d1d9f004c1b8d3594fd30da1024, scope.project.id=a3eb71934e924f9e801e61e599cba492, user.id=31e4ee38bf8148f4a4dd9f56097514a1) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:31:59.515476 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe543f-4cd4-4c2b-8df9-5a2e6b485f84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:31:59.517146 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe543f-4cd4-4c2b-8df9-5a2e6b485f84 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:31:59.523519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 88/180] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:31:59 2022] GET /identity/v3/role_assignments?user.id=31e4ee38bf8148f4a4dd9f56097514a1&scope.project.id=a3eb71934e924f9e801e61e599cba492&role.id=b3244d1d9f004c1b8d3594fd30da1024 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:31:59.840570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718a74b-ded2-4606-b666-b45e2bea4d0f None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:59.841127 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718a74b-ded2-4606-b666-b45e2bea4d0f None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:59.841385 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718a74b-ded2-4606-b666-b45e2bea4d0f None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:59.843409 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 93/181] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:31:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:31:59.848613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f871e24-05b3-4cd1-b59a-5a782650dc86 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:31:59.849376 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f871e24-05b3-4cd1-b59a-5a782650dc86 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:31:59.849667 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f871e24-05b3-4cd1-b59a-5a782650dc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:31:59.870560 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-8f871e24-05b3-4cd1-b59a-5a782650dc86 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:31:59.871755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-8f871e24-05b3-4cd1-b59a-5a782650dc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:31:59.908567 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:31:59.944131 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 89/182] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:31:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:31:59.969538 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718a74b-ded2-4606-b666-b45e2bea4d0f None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:00.075223 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4defbb-dce7-4ee0-9f24-6cfb70c17edc None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:00.076078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4defbb-dce7-4ee0-9f24-6cfb70c17edc None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:00.076404 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4defbb-dce7-4ee0-9f24-6cfb70c17edc None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:00.077944 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 90/183] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:00.082519 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:00.083874 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83c916-8c48-4786-b599-36c444c4e789 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:00.084144 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83c916-8c48-4786-b599-36c444c4e789 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:00.084425 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83c916-8c48-4786-b599-36c444c4e789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:00.103257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-ff83c916-8c48-4786-b599-36c444c4e789 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:00.103983 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718a74b-ded2-4606-b666-b45e2bea4d0f None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:00.104513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-ff83c916-8c48-4786-b599-36c444c4e789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:00.141108 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5c05e-b204-4cc7-85c4-6e06fe203897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:00.142195 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5c05e-b204-4cc7-85c4-6e06fe203897 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:00.142680 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5c05e-b204-4cc7-85c4-6e06fe203897 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:00.143126 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5c05e-b204-4cc7-85c4-6e06fe203897 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:00.154564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e5c05e-b204-4cc7-85c4-6e06fe203897 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:00.162765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-d2e5c05e-b204-4cc7-85c4-6e06fe203897 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:32:00.162765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:00.162765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:00.162765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:00.162765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:00.162765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:00.162765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:00.162765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:00.162765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:00.162765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:00.162765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:00.162765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:00.162765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:00.162765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:00.162765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:00.162765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:00.163646 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:00.163646 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:00.163646 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:00.163646 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:00.163646 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:00.163646 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:00.163646 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:00.163646 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:00.163646 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:00.163646 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:00.163646 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:00.163646 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:00.163646 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:00.163646 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:00.163646 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:00.163646 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:00.164427 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:00.164427 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:00.164427 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:00.164427 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:00.164427 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:00.164427 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:00.164427 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:00.164427 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:00.164427 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:00.164427 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:00.164427 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:32:00.164427 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:00.167357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 94/184] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:31:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:00.173444 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5c05e-b204-4cc7-85c4-6e06fe203897 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:00.176992 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 91/185] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 13:32:00.200819 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff83c916-8c48-4786-b599-36c444c4e789 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:00.278582 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5c05e-b204-4cc7-85c4-6e06fe203897 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:00.308365 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff83c916-8c48-4786-b599-36c444c4e789 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:00.311151 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:00.319650 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d3711-c75d-4817-a6af-9711bf4b0d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:00.320332 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d3711-c75d-4817-a6af-9711bf4b0d77 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:00.320623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d3711-c75d-4817-a6af-9711bf4b0d77 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:00.320905 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d3711-c75d-4817-a6af-9711bf4b0d77 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:00.321706 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d3711-c75d-4817-a6af-9711bf4b0d77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:00.322018 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d3711-c75d-4817-a6af-9711bf4b0d77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:00.323387 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d3711-c75d-4817-a6af-9711bf4b0d77 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:00.332167 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 95/186] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:32:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:00.334873 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669feb5a-54eb-4f2a-a4c6-40a551818b0c None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:00.335528 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669feb5a-54eb-4f2a-a4c6-40a551818b0c None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:00.335892 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669feb5a-54eb-4f2a-a4c6-40a551818b0c None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:00.337541 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 96/187] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:00.341876 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669feb5a-54eb-4f2a-a4c6-40a551818b0c None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:00.346322 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:00.353076 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a2d53-a992-4f90-b320-c680a74c6b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:00.353963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a2d53-a992-4f90-b320-c680a74c6b1e None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:00.354195 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a2d53-a992-4f90-b320-c680a74c6b1e None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:00.354473 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a2d53-a992-4f90-b320-c680a74c6b1e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:00.365799 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a2d53-a992-4f90-b320-c680a74c6b1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:00.374922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-496a2d53-a992-4f90-b320-c680a74c6b1e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 13:32:00.374922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:00.374922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:00.374922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:00.374922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:00.374922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:00.374922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:00.374922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:00.374922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:00.374922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:00.374922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:00.374922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:00.374922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:00.374922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:00.374922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:00.374922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:00.375812 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:00.375812 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:00.375812 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:00.375812 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:00.375812 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:00.375812 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:00.375812 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:00.375812 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:00.375812 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:00.375812 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:00.375812 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:00.375812 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:00.375812 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:00.375812 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:00.375812 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:00.375812 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:00.377669 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:00.377669 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:00.377669 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:00.377669 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:00.377669 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:00.377669 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:00.377669 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:00.377669 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:00.377669 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:00.377669 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:00.377669 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 13:32:00.377669 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:00.377669 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 92/188] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jul 6 13:32:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:00.381993 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b4164-a099-4c98-b1e8-e1c3f624939a None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:00.382285 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b4164-a099-4c98-b1e8-e1c3f624939a None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:00.382680 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b4164-a099-4c98-b1e8-e1c3f624939a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:00.400046 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-cb3b4164-a099-4c98-b1e8-e1c3f624939a None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:00.401267 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-cb3b4164-a099-4c98-b1e8-e1c3f624939a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:00.455856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669feb5a-54eb-4f2a-a4c6-40a551818b0c None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:00.477397 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 93/189] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:00.478660 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b4164-a099-4c98-b1e8-e1c3f624939a None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:00.495916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e70e2-4bc6-4a96-abea-c3423824463f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:00.496795 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e70e2-4bc6-4a96-abea-c3423824463f None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:00.497107 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e70e2-4bc6-4a96-abea-c3423824463f None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:00.498173 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e70e2-4bc6-4a96-abea-c3423824463f None admin] PATH_INFO: `/v3/users/79482c0d85544304a916213e9d65d108` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:00.515809 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e70e2-4bc6-4a96-abea-c3423824463f None admin] RBAC: Authorizing `identity:get_user(user_id=79482c0d85544304a916213e9d65d108)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:00.517287 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e70e2-4bc6-4a96-abea-c3423824463f None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:00.518970 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 97/190] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:32:00 2022] GET /identity/v3/users/79482c0d85544304a916213e9d65d108 => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:00.519732 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e70e2-4bc6-4a96-abea-c3423824463f None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:00.581364 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:00.588943 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:00.601415 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b4164-a099-4c98-b1e8-e1c3f624939a None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:00.622450 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e70e2-4bc6-4a96-abea-c3423824463f None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:00.635780 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfd6ef-f76c-48e9-8d22-0bcf5911935e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:00.636538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfd6ef-f76c-48e9-8d22-0bcf5911935e None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:00.636824 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfd6ef-f76c-48e9-8d22-0bcf5911935e None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:00.637351 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfd6ef-f76c-48e9-8d22-0bcf5911935e None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:00.638422 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cfd6ef-f76c-48e9-8d22-0bcf5911935e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:00.638938 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cfd6ef-f76c-48e9-8d22-0bcf5911935e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:00.640453 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cfd6ef-f76c-48e9-8d22-0bcf5911935e None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:00.648934 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 94/191] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:32:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:00.649580 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfd6ef-f76c-48e9-8d22-0bcf5911935e None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:00.672671 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c442686-dce2-4178-8eaf-2761467dc437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:00.673500 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c442686-dce2-4178-8eaf-2761467dc437 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:00.673780 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c442686-dce2-4178-8eaf-2761467dc437 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:00.674307 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c442686-dce2-4178-8eaf-2761467dc437 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:00.675475 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c442686-dce2-4178-8eaf-2761467dc437 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:00.676924 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c442686-dce2-4178-8eaf-2761467dc437 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:00.678881 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 98/192] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:00.679703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c442686-dce2-4178-8eaf-2761467dc437 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:00.765434 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfd6ef-f76c-48e9-8d22-0bcf5911935e None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:00.785563 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c442686-dce2-4178-8eaf-2761467dc437 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:00.801867 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7aab8-b2cf-42a3-bd29-5ca69d4032b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:00.802527 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7aab8-b2cf-42a3-bd29-5ca69d4032b1 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:00.802814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7aab8-b2cf-42a3-bd29-5ca69d4032b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:00.803105 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7aab8-b2cf-42a3-bd29-5ca69d4032b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:00.816606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:00.823280 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af8aa9-378e-4f46-aa91-1481d88e3e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:00.824440 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af8aa9-378e-4f46-aa91-1481d88e3e00 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:00.824721 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af8aa9-378e-4f46-aa91-1481d88e3e00 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:00.824991 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af8aa9-378e-4f46-aa91-1481d88e3e00 None admin] PATH_INFO: `/v3/users/79482c0d85544304a916213e9d65d108` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:00.826299 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af8aa9-378e-4f46-aa91-1481d88e3e00 None admin] RBAC: Authorizing `identity:get_user(user_id=79482c0d85544304a916213e9d65d108)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:00.828455 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af8aa9-378e-4f46-aa91-1481d88e3e00 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:00.829211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:00.831124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 99/193] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:32:00 2022] GET /identity/v3/users/79482c0d85544304a916213e9d65d108 => generated 283 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:00.836162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af8aa9-378e-4f46-aa91-1481d88e3e00 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:00.879216 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e7aab8-b2cf-42a3-bd29-5ca69d4032b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:00.880597 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e7aab8-b2cf-42a3-bd29-5ca69d4032b1 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:00.942251 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af8aa9-378e-4f46-aa91-1481d88e3e00 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:00.983987 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267bb32-25f6-4ab5-9e97-a403cfeb3aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:00.984667 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267bb32-25f6-4ab5-9e97-a403cfeb3aa8 None admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:00.984952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267bb32-25f6-4ab5-9e97-a403cfeb3aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:00.985319 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267bb32-25f6-4ab5-9e97-a403cfeb3aa8 None admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:00.985984 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4267bb32-25f6-4ab5-9e97-a403cfeb3aa8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:00.987423 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4267bb32-25f6-4ab5-9e97-a403cfeb3aa8 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:00.996141 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 95/194] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:32:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:32:00.996141 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7aab8-b2cf-42a3-bd29-5ca69d4032b1 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:01.008862 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 100/195] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jul 6 13:32:00 2022] POST /identity/v3/users => generated 297 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:32:01.009580 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267bb32-25f6-4ab5-9e97-a403cfeb3aa8 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:01.073952 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:01.079482 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:01.129491 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7aab8-b2cf-42a3-bd29-5ca69d4032b1 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:01.134169 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267bb32-25f6-4ab5-9e97-a403cfeb3aa8 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:01.171702 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc65c1b7-2d49-4a97-ab6b-1671216fd8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:01.172367 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65c1b7-2d49-4a97-ab6b-1671216fd8a4 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:01.172674 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65c1b7-2d49-4a97-ab6b-1671216fd8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:01.172930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65c1b7-2d49-4a97-ab6b-1671216fd8a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:01.175849 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa148dc-6c30-4115-9ac2-5946e00c0a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:01.176613 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa148dc-6c30-4115-9ac2-5946e00c0a32 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:01.176902 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa148dc-6c30-4115-9ac2-5946e00c0a32 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:01.177206 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa148dc-6c30-4115-9ac2-5946e00c0a32 None admin] PATH_INFO: `/v3/projects/acd1ac1812eb4faeb54c342d511d42de` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:01.178404 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa148dc-6c30-4115-9ac2-5946e00c0a32 None admin] RBAC: Authorizing `identity:get_project(project_id=acd1ac1812eb4faeb54c342d511d42de)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:01.179849 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa148dc-6c30-4115-9ac2-5946e00c0a32 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:01.181761 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 101/196] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:32:01 2022] GET /identity/v3/projects/acd1ac1812eb4faeb54c342d511d42de => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:01.187330 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa148dc-6c30-4115-9ac2-5946e00c0a32 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:01.256726 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc65c1b7-2d49-4a97-ab6b-1671216fd8a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:01.258247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc65c1b7-2d49-4a97-ab6b-1671216fd8a4 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:01.321493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa148dc-6c30-4115-9ac2-5946e00c0a32 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:01.333976 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:01.352439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:01.374345 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b169b880-2d43-4c4f-a4b7-7577bd841f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:01.375369 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b169b880-2d43-4c4f-a4b7-7577bd841f19 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:01.375847 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b169b880-2d43-4c4f-a4b7-7577bd841f19 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:01.376335 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b169b880-2d43-4c4f-a4b7-7577bd841f19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:01.377424 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b169b880-2d43-4c4f-a4b7-7577bd841f19 None admin] RBAC: Adding query filter params (role.id=795daa2c78354fd48e4a741e1182cc98, scope.project.id=acd1ac1812eb4faeb54c342d511d42de, user.id=79482c0d85544304a916213e9d65d108) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:01.377806 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b169b880-2d43-4c4f-a4b7-7577bd841f19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:01.379325 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b169b880-2d43-4c4f-a4b7-7577bd841f19 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:01.386184 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 102/197] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:32:01 2022] GET /identity/v3/role_assignments?user.id=79482c0d85544304a916213e9d65d108&scope.project.id=acd1ac1812eb4faeb54c342d511d42de&role.id=795daa2c78354fd48e4a741e1182cc98 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:01.387302 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee02b1-5590-4801-9697-80adc80e4704 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:01.389223 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee02b1-5590-4801-9697-80adc80e4704 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:01.389547 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee02b1-5590-4801-9697-80adc80e4704 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:01.391379 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 103/198] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:01.397310 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3ec4d-9974-49fa-a236-81fb49586507 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:01.397597 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3ec4d-9974-49fa-a236-81fb49586507 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:01.397940 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3ec4d-9974-49fa-a236-81fb49586507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:01.402767 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 96/199] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:32:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:32:01.408061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc65c1b7-2d49-4a97-ab6b-1671216fd8a4 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:01.418378 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-4db3ec4d-9974-49fa-a236-81fb49586507 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:01.420134 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-4db3ec4d-9974-49fa-a236-81fb49586507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:01.503635 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 104/200] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 13:32:01.530173 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3ec4d-9974-49fa-a236-81fb49586507 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:01.538789 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc65c1b7-2d49-4a97-ab6b-1671216fd8a4 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:01.584799 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793acae6-5c0e-42e3-bbe6-112674970434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:01.586079 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793acae6-5c0e-42e3-bbe6-112674970434 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:01.586506 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793acae6-5c0e-42e3-bbe6-112674970434 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:01.587057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793acae6-5c0e-42e3-bbe6-112674970434 None admin] PATH_INFO: `/v3/projects/acd1ac1812eb4faeb54c342d511d42de` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:01.588442 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793acae6-5c0e-42e3-bbe6-112674970434 None admin] RBAC: Authorizing `identity:get_project(project_id=acd1ac1812eb4faeb54c342d511d42de)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:01.590728 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793acae6-5c0e-42e3-bbe6-112674970434 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:01.593982 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 97/201] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:32:01 2022] GET /identity/v3/projects/acd1ac1812eb4faeb54c342d511d42de => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:01.601799 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793acae6-5c0e-42e3-bbe6-112674970434 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:01.657445 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:01.701410 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3ec4d-9974-49fa-a236-81fb49586507 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:01.721710 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:01.744732 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d88d8-9e30-408e-baf5-fc2e998e242d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:01.745915 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d88d8-9e30-408e-baf5-fc2e998e242d None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:01.746396 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d88d8-9e30-408e-baf5-fc2e998e242d None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:01.746863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d88d8-9e30-408e-baf5-fc2e998e242d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:01.759719 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d88d8-9e30-408e-baf5-fc2e998e242d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:01.768221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-125d88d8-9e30-408e-baf5-fc2e998e242d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 13:32:01.768221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:01.768221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:01.768221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:01.768221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:01.768221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:01.768221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:01.768221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:01.768221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:01.768221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:01.768221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:01.768221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:01.768221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:01.768221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:01.768221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:01.768221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:01.769128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:01.769128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:01.769128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:01.769128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:01.769128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:01.769128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:01.769128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:01.769128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:01.769128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:01.769128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:01.769128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:01.769128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:01.769128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:01.769128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:01.769128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:01.769128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:01.769985 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:01.769985 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:01.769985 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:01.769985 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:01.769985 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:01.769985 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:01.769985 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:01.769985 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:01.769985 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:01.769985 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:01.769985 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 13:32:01.769985 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:01.773216 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 105/202] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:32:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:01.781985 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d88d8-9e30-408e-baf5-fc2e998e242d None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:01.807138 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793acae6-5c0e-42e3-bbe6-112674970434 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:01.844274 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44582807-4486-408f-a9de-19206c451e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:01.845875 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44582807-4486-408f-a9de-19206c451e38 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:01.849243 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44582807-4486-408f-a9de-19206c451e38 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:01.849556 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44582807-4486-408f-a9de-19206c451e38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:01.850417 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44582807-4486-408f-a9de-19206c451e38 None admin] RBAC: Adding query filter params (role.id=534400e7b28546a8a04c08f825c28566, scope.project.id=acd1ac1812eb4faeb54c342d511d42de, user.id=79482c0d85544304a916213e9d65d108) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:01.850733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44582807-4486-408f-a9de-19206c451e38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:01.852187 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44582807-4486-408f-a9de-19206c451e38 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:01.873119 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 98/203] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:32:01 2022] GET /identity/v3/role_assignments?user.id=79482c0d85544304a916213e9d65d108&scope.project.id=acd1ac1812eb4faeb54c342d511d42de&role.id=534400e7b28546a8a04c08f825c28566 => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:01.918454 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d88d8-9e30-408e-baf5-fc2e998e242d None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:01.958833 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3711b0-cd53-4be8-8857-e4590e48ca0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:01.960351 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3711b0-cd53-4be8-8857-e4590e48ca0b None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:01.960897 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3711b0-cd53-4be8-8857-e4590e48ca0b None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:01.961394 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3711b0-cd53-4be8-8857-e4590e48ca0b None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:01.962392 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3711b0-cd53-4be8-8857-e4590e48ca0b None admin] RBAC: Adding query filter params (name=reader) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:01.962906 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3711b0-cd53-4be8-8857-e4590e48ca0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:01.964606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3711b0-cd53-4be8-8857-e4590e48ca0b None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:01.975373 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 106/204] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jul 6 13:32:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:01.981199 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44582807-4486-408f-a9de-19206c451e38 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:02.115027 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:02.137248 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44582807-4486-408f-a9de-19206c451e38 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:02.181060 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59907c3a-fe5b-4df7-8211-81cf61d63bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:02.181874 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59907c3a-fe5b-4df7-8211-81cf61d63bc4 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:02.183129 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59907c3a-fe5b-4df7-8211-81cf61d63bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:02.183407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59907c3a-fe5b-4df7-8211-81cf61d63bc4 None admin] PATH_INFO: `/v3/users/33e0cd5bda0d46059c4a848ccf7a6385` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:02.189839 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59907c3a-fe5b-4df7-8211-81cf61d63bc4 None admin] RBAC: Authorizing `identity:get_user(user_id=33e0cd5bda0d46059c4a848ccf7a6385)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:02.191316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59907c3a-fe5b-4df7-8211-81cf61d63bc4 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:02.193070 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 99/205] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:32:01 2022] GET /identity/v3/users/33e0cd5bda0d46059c4a848ccf7a6385 => generated 289 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:02.198520 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3711b0-cd53-4be8-8857-e4590e48ca0b None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:02.264800 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:02.355581 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3711b0-cd53-4be8-8857-e4590e48ca0b None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:02.410292 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea1803-97b7-43a2-aee2-6621474747db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:02.410957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea1803-97b7-43a2-aee2-6621474747db None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:02.411238 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea1803-97b7-43a2-aee2-6621474747db None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:02.411515 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea1803-97b7-43a2-aee2-6621474747db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:02.506675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adea1803-97b7-43a2-aee2-6621474747db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:02.508133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adea1803-97b7-43a2-aee2-6621474747db None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:02.558592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:02.627038 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 107/206] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:32:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:32:02.632664 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59907c3a-fe5b-4df7-8211-81cf61d63bc4 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:02.757967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59907c3a-fe5b-4df7-8211-81cf61d63bc4 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:02.792241 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab310a-2697-4eca-a80e-d7e73cc572e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:02.793105 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab310a-2697-4eca-a80e-d7e73cc572e1 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:02.793390 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab310a-2697-4eca-a80e-d7e73cc572e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:02.793665 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab310a-2697-4eca-a80e-d7e73cc572e1 None admin] PATH_INFO: `/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:02.795029 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab310a-2697-4eca-a80e-d7e73cc572e1 None admin] RBAC: Authorizing `identity:get_project(project_id=202d7ba5c7f34872aa3ab82e5d2b2f93)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:02.796495 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab310a-2697-4eca-a80e-d7e73cc572e1 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:02.803343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 100/207] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:32:02 2022] GET /identity/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:02.804688 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea1803-97b7-43a2-aee2-6621474747db None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:02.943008 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea1803-97b7-43a2-aee2-6621474747db None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:02.990513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c391b5a-9cd4-4b28-b243-7a8955dc54df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:02.991281 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c391b5a-9cd4-4b28-b243-7a8955dc54df None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:02.991493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c391b5a-9cd4-4b28-b243-7a8955dc54df None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:02.991772 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c391b5a-9cd4-4b28-b243-7a8955dc54df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:02.992611 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c391b5a-9cd4-4b28-b243-7a8955dc54df None admin] RBAC: Adding query filter params (role.id=65ecdcc5a9d0487ba529e54c626e152f, scope.project.id=202d7ba5c7f34872aa3ab82e5d2b2f93, user.id=33e0cd5bda0d46059c4a848ccf7a6385) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:02.992921 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c391b5a-9cd4-4b28-b243-7a8955dc54df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:02.994376 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c391b5a-9cd4-4b28-b243-7a8955dc54df None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:03.005558 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 108/208] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:32:02 2022] GET /identity/v3/role_assignments?user.id=33e0cd5bda0d46059c4a848ccf7a6385&scope.project.id=202d7ba5c7f34872aa3ab82e5d2b2f93&role.id=65ecdcc5a9d0487ba529e54c626e152f => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:03.485655 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322024f-e24f-46ce-abf1-dacd4d9b2349 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:03.486155 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322024f-e24f-46ce-abf1-dacd4d9b2349 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:03.486229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322024f-e24f-46ce-abf1-dacd4d9b2349 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:03.487578 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 101/209] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:03.493240 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8c14f-5e01-494d-b8cc-89e3a9a2b197 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:03.493531 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8c14f-5e01-494d-b8cc-89e3a9a2b197 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:03.493814 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8c14f-5e01-494d-b8cc-89e3a9a2b197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:03.499548 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:03.512497 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-09c8c14f-5e01-494d-b8cc-89e3a9a2b197 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:03.514371 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-09c8c14f-5e01-494d-b8cc-89e3a9a2b197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:03.588396 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 109/210] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:03.611958 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322024f-e24f-46ce-abf1-dacd4d9b2349 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:03.639493 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:03.720681 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322024f-e24f-46ce-abf1-dacd4d9b2349 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:03.755721 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d4174-3e2a-4d60-a493-5408a6b04870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:03.756521 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d4174-3e2a-4d60-a493-5408a6b04870 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:03.756805 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d4174-3e2a-4d60-a493-5408a6b04870 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:03.757118 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d4174-3e2a-4d60-a493-5408a6b04870 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:03.768411 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d4174-3e2a-4d60-a493-5408a6b04870 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:03.776089 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-418d4174-3e2a-4d60-a493-5408a6b04870 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:32:03.776089 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:03.776089 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:03.776089 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:03.776089 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:03.776089 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:03.776089 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:03.776089 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:03.776089 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:03.776089 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:03.776089 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:03.776089 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:03.776089 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:03.776089 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:03.776089 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:03.776089 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:03.776972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:03.776972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:03.776972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:03.776972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:03.776972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:03.776972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:03.776972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:03.776972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:03.776972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:03.776972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:03.776972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:03.776972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:03.776972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:03.776972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:03.776972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:03.776972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:03.777772 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:03.777772 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:03.777772 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:03.777772 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:03.777772 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:03.777772 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:03.777772 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:03.777772 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:03.777772 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:03.777772 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:03.777772 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:32:03.777772 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:03.777772 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 102/211] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:32:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:03.785033 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8c14f-5e01-494d-b8cc-89e3a9a2b197 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:03.810492 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b19aca-6a77-4041-bed2-ddb5a26fe84a None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:03.810955 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b19aca-6a77-4041-bed2-ddb5a26fe84a None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:03.811214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b19aca-6a77-4041-bed2-ddb5a26fe84a None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:03.812662 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 103/212] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:32:03.817850 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f2428-5f55-4fc0-8cc8-ef922bb79b28 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:03.818145 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f2428-5f55-4fc0-8cc8-ef922bb79b28 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:03.818433 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f2428-5f55-4fc0-8cc8-ef922bb79b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:03.837227 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-4c8f2428-5f55-4fc0-8cc8-ef922bb79b28 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:03.838578 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-4c8f2428-5f55-4fc0-8cc8-ef922bb79b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:03.908253 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8c14f-5e01-494d-b8cc-89e3a9a2b197 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:03.912111 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 104/213] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 13:32:03.917265 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ea3ab-3d71-4516-963d-8139ff9fcefa None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:03.917767 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ea3ab-3d71-4516-963d-8139ff9fcefa None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:03.918331 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ea3ab-3d71-4516-963d-8139ff9fcefa None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:03.920010 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 105/214] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:03.925704 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e11de-2ffc-4de6-97d8-a29245acfb52 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:03.925991 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e11de-2ffc-4de6-97d8-a29245acfb52 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:03.926369 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e11de-2ffc-4de6-97d8-a29245acfb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:03.943174 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:03.944918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-315e11de-2ffc-4de6-97d8-a29245acfb52 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:03.946488 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-315e11de-2ffc-4de6-97d8-a29245acfb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:03.948674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbc463-26cf-43b8-80ab-5e6d231bc4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:03.949385 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbc463-26cf-43b8-80ab-5e6d231bc4e7 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:03.949672 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbc463-26cf-43b8-80ab-5e6d231bc4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:03.949952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbc463-26cf-43b8-80ab-5e6d231bc4e7 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:03.951278 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cbc463-26cf-43b8-80ab-5e6d231bc4e7 None admin] RBAC: Adding query filter params (name=member) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:03.951773 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cbc463-26cf-43b8-80ab-5e6d231bc4e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:03.953185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cbc463-26cf-43b8-80ab-5e6d231bc4e7 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:03.955496 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:03.966786 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 110/215] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jul 6 13:32:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:03.967512 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbc463-26cf-43b8-80ab-5e6d231bc4e7 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:04.021389 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 106/216] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 13:32:04.021389 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e11de-2ffc-4de6-97d8-a29245acfb52 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:04.079499 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbc463-26cf-43b8-80ab-5e6d231bc4e7 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:04.114722 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41146970-d040-43f2-af03-768cbb5e8385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:04.115464 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41146970-d040-43f2-af03-768cbb5e8385 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:04.115753 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41146970-d040-43f2-af03-768cbb5e8385 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:04.116032 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41146970-d040-43f2-af03-768cbb5e8385 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:04.117250 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41146970-d040-43f2-af03-768cbb5e8385 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:04.118695 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41146970-d040-43f2-af03-768cbb5e8385 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:04.120535 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 111/217] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:04.122837 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41146970-d040-43f2-af03-768cbb5e8385 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:04.133591 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e11de-2ffc-4de6-97d8-a29245acfb52 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:04.171879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d1f58-e813-4f03-be31-81562371e139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:04.172764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d1f58-e813-4f03-be31-81562371e139 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:04.173071 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d1f58-e813-4f03-be31-81562371e139 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:04.173349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d1f58-e813-4f03-be31-81562371e139 None admin] PATH_INFO: `/v3/users/7ac97a3a7ed54185a876e936bf03bfb1` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:04.184772 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:04.187611 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d1f58-e813-4f03-be31-81562371e139 None admin] RBAC: Authorizing `identity:get_user(user_id=7ac97a3a7ed54185a876e936bf03bfb1)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:04.189264 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d1f58-e813-4f03-be31-81562371e139 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:04.191630 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 107/218] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:32:04 2022] GET /identity/v3/users/7ac97a3a7ed54185a876e936bf03bfb1 => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:04.192785 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d1f58-e813-4f03-be31-81562371e139 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:04.198712 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:04.245621 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41146970-d040-43f2-af03-768cbb5e8385 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:04.279419 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212dd4ca-65c6-40cb-8e85-df88edb4970e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:04.280209 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212dd4ca-65c6-40cb-8e85-df88edb4970e None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:04.280489 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212dd4ca-65c6-40cb-8e85-df88edb4970e None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:04.280768 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212dd4ca-65c6-40cb-8e85-df88edb4970e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:04.292040 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212dd4ca-65c6-40cb-8e85-df88edb4970e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:04.300086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-212dd4ca-65c6-40cb-8e85-df88edb4970e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:32:04.300086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:04.300086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:04.300086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:04.300086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:04.300086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:04.300086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:04.300086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:04.300086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:04.300086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:04.300086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:04.300086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:04.300086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:04.300086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:04.300086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:04.300086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:04.301475 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:04.301475 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:04.301475 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:04.301475 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:04.301475 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:04.301475 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:04.301475 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:04.301475 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:04.301475 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:04.301475 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:04.301475 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:04.301475 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:04.301475 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:04.301475 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:04.301475 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:04.301475 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:04.302302 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:04.302302 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:04.302302 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:04.302302 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:04.302302 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:04.302302 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:04.302302 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:04.302302 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:04.302302 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:04.302302 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:04.302302 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:32:04.302302 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:04.302302 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 112/219] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:32:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:04.305333 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212dd4ca-65c6-40cb-8e85-df88edb4970e None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:04.308283 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d1f58-e813-4f03-be31-81562371e139 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:04.346137 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0a8a4-f505-4292-b126-1c9ff8462f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:04.346929 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0a8a4-f505-4292-b126-1c9ff8462f15 None admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:04.347312 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0a8a4-f505-4292-b126-1c9ff8462f15 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:04.347658 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0a8a4-f505-4292-b126-1c9ff8462f15 None admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:04.348380 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c0a8a4-f505-4292-b126-1c9ff8462f15 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:04.349862 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c0a8a4-f505-4292-b126-1c9ff8462f15 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:04.371768 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 108/220] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jul 6 13:32:04 2022] POST /identity/v3/users => generated 297 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:32:04.373092 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0a8a4-f505-4292-b126-1c9ff8462f15 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:04.421297 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212dd4ca-65c6-40cb-8e85-df88edb4970e None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:04.448823 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:04.456699 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:04.463115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b537026-7eea-4b65-b531-89d69fce2536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:04.463918 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b537026-7eea-4b65-b531-89d69fce2536 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:04.464214 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b537026-7eea-4b65-b531-89d69fce2536 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:04.464498 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b537026-7eea-4b65-b531-89d69fce2536 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:04.506773 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0a8a4-f505-4292-b126-1c9ff8462f15 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:04.545085 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bcf8a4-8b4e-47c4-a6b3-070c1a494e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:04.546144 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bcf8a4-8b4e-47c4-a6b3-070c1a494e22 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:04.546526 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bcf8a4-8b4e-47c4-a6b3-070c1a494e22 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:04.546905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bcf8a4-8b4e-47c4-a6b3-070c1a494e22 None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:04.547782 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bcf8a4-8b4e-47c4-a6b3-070c1a494e22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:04.548164 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bcf8a4-8b4e-47c4-a6b3-070c1a494e22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:04.551892 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bcf8a4-8b4e-47c4-a6b3-070c1a494e22 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:04.552014 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b537026-7eea-4b65-b531-89d69fce2536 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:04.552750 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b537026-7eea-4b65-b531-89d69fce2536 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:04.561168 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 109/221] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:32:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:04.566712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bcf8a4-8b4e-47c4-a6b3-070c1a494e22 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:04.675900 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 113/222] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:32:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:32:04.681496 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff338844-e6ce-4ed5-846f-3480ec6b70de None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:04.681985 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff338844-e6ce-4ed5-846f-3480ec6b70de None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:04.682416 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff338844-e6ce-4ed5-846f-3480ec6b70de None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:04.683977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 114/223] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:04.687294 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff338844-e6ce-4ed5-846f-3480ec6b70de None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:04.696603 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bcf8a4-8b4e-47c4-a6b3-070c1a494e22 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:04.716318 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:04.719191 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:04.746803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b45db49-b1eb-45e1-91a6-b93ba8a652f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:04.748569 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45db49-b1eb-45e1-91a6-b93ba8a652f6 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:04.748952 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45db49-b1eb-45e1-91a6-b93ba8a652f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:04.749376 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45db49-b1eb-45e1-91a6-b93ba8a652f6 None admin] PATH_INFO: `/v3/users/79482c0d85544304a916213e9d65d108` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:04.750829 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b45db49-b1eb-45e1-91a6-b93ba8a652f6 None admin] RBAC: Authorizing `identity:get_user(user_id=79482c0d85544304a916213e9d65d108)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:04.752434 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b45db49-b1eb-45e1-91a6-b93ba8a652f6 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:04.754533 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 110/224] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:32:04 2022] GET /identity/v3/users/79482c0d85544304a916213e9d65d108 => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:04.760260 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6d259-bbec-44e8-9408-9e276e4baa4f None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:04.760647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6d259-bbec-44e8-9408-9e276e4baa4f None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:04.761092 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6d259-bbec-44e8-9408-9e276e4baa4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:04.779555 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-31a6d259-bbec-44e8-9408-9e276e4baa4f None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:04.781182 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-31a6d259-bbec-44e8-9408-9e276e4baa4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:04.808693 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff338844-e6ce-4ed5-846f-3480ec6b70de None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:04.848531 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d92606-fb01-492b-b206-8d3c29785d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:04.849773 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d92606-fb01-492b-b206-8d3c29785d7f None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:04.850312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d92606-fb01-492b-b206-8d3c29785d7f None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:04.850787 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d92606-fb01-492b-b206-8d3c29785d7f None admin] PATH_INFO: `/v3/projects/acd1ac1812eb4faeb54c342d511d42de` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:04.852311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d92606-fb01-492b-b206-8d3c29785d7f None admin] RBAC: Authorizing `identity:get_project(project_id=acd1ac1812eb4faeb54c342d511d42de)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:04.854125 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d92606-fb01-492b-b206-8d3c29785d7f None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:04.856218 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 115/225] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:32:04 2022] GET /identity/v3/projects/acd1ac1812eb4faeb54c342d511d42de => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:04.862282 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d92606-fb01-492b-b206-8d3c29785d7f None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:04.863274 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 111/226] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:04.864035 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6d259-bbec-44e8-9408-9e276e4baa4f None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:04.989685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6d259-bbec-44e8-9408-9e276e4baa4f None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:04.990808 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:05.008382 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:05.010053 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d92606-fb01-492b-b206-8d3c29785d7f None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:05.031385 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4832f51b-0fd6-4170-aec8-6f7834578745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:05.032477 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832f51b-0fd6-4170-aec8-6f7834578745 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:05.032963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832f51b-0fd6-4170-aec8-6f7834578745 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:05.033469 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832f51b-0fd6-4170-aec8-6f7834578745 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:05.034735 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4832f51b-0fd6-4170-aec8-6f7834578745 None admin] RBAC: Adding query filter params (role.id=b3244d1d9f004c1b8d3594fd30da1024, scope.project.id=acd1ac1812eb4faeb54c342d511d42de, user.id=7ac97a3a7ed54185a876e936bf03bfb1) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:05.035056 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4832f51b-0fd6-4170-aec8-6f7834578745 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:05.036537 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4832f51b-0fd6-4170-aec8-6f7834578745 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:05.042898 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 112/227] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:32:04 2022] GET /identity/v3/role_assignments?user.id=7ac97a3a7ed54185a876e936bf03bfb1&scope.project.id=acd1ac1812eb4faeb54c342d511d42de&role.id=b3244d1d9f004c1b8d3594fd30da1024 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:05.050323 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4832f51b-0fd6-4170-aec8-6f7834578745 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:05.059059 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a27fdd-2d40-4cea-a798-503443604c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:05.059957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a27fdd-2d40-4cea-a798-503443604c4c None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:05.060244 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a27fdd-2d40-4cea-a798-503443604c4c None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:05.060623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a27fdd-2d40-4cea-a798-503443604c4c None admin] PATH_INFO: `/v3/projects/acd1ac1812eb4faeb54c342d511d42de` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:05.066011 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a27fdd-2d40-4cea-a798-503443604c4c None admin] RBAC: Authorizing `identity:get_project(project_id=acd1ac1812eb4faeb54c342d511d42de)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:05.069034 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a27fdd-2d40-4cea-a798-503443604c4c None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:05.069709 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 116/228] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:32:04 2022] GET /identity/v3/projects/acd1ac1812eb4faeb54c342d511d42de => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:05.075968 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a27fdd-2d40-4cea-a798-503443604c4c None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:05.213169 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4832f51b-0fd6-4170-aec8-6f7834578745 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:05.256316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa177437-d353-4d15-a6b5-22b62bde9ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:05.257205 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa177437-d353-4d15-a6b5-22b62bde9ccc None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:05.257494 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa177437-d353-4d15-a6b5-22b62bde9ccc None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:05.257789 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa177437-d353-4d15-a6b5-22b62bde9ccc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:05.259056 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a27fdd-2d40-4cea-a798-503443604c4c None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:05.286504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa177437-d353-4d15-a6b5-22b62bde9ccc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:05.301872 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-aa177437-d353-4d15-a6b5-22b62bde9ccc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 13:32:05.301872 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:05.301872 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:05.301872 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:05.301872 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:05.301872 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:05.301872 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:05.301872 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:05.301872 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:05.301872 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:05.301872 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:05.301872 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:05.301872 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:05.301872 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:05.301872 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:05.301872 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:05.302806 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:05.302806 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:05.302806 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:05.302806 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:05.302806 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:05.302806 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:05.302806 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:05.302806 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:05.302806 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:05.302806 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:05.302806 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:05.302806 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:05.302806 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:05.302806 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:05.302806 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:05.302806 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:05.303606 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:05.303606 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:05.303606 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:05.303606 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:05.303606 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:05.303606 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:05.303606 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:05.303606 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:05.303606 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:05.303606 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:05.303606 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 13:32:05.303606 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:05.303606 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 113/229] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jul 6 13:32:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:05.311232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc10b5-c464-466f-afa9-9a0743a3fb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:05.311795 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa177437-d353-4d15-a6b5-22b62bde9ccc None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:05.312150 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc10b5-c464-466f-afa9-9a0743a3fb2a None admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:05.312435 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc10b5-c464-466f-afa9-9a0743a3fb2a None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:05.312715 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc10b5-c464-466f-afa9-9a0743a3fb2a None admin] PATH_INFO: `/v3/projects/acd1ac1812eb4faeb54c342d511d42de/users/79482c0d85544304a916213e9d65d108/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:05.314908 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc10b5-c464-466f-afa9-9a0743a3fb2a None admin] RBAC: Authorizing `identity:create_grant(project_id=acd1ac1812eb4faeb54c342d511d42de, user_id=79482c0d85544304a916213e9d65d108, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:05.316332 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc10b5-c464-466f-afa9-9a0743a3fb2a None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:05.341390 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 117/230] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jul 6 13:32:05 2022] PUT /identity/v3/projects/acd1ac1812eb4faeb54c342d511d42de/users/79482c0d85544304a916213e9d65d108/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:32:05.405397 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:05.464463 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa177437-d353-4d15-a6b5-22b62bde9ccc None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:05.510467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b88c3-2783-45fe-a2a1-00d00815b7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:05.511066 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b88c3-2783-45fe-a2a1-00d00815b7db None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:05.511355 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b88c3-2783-45fe-a2a1-00d00815b7db None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:05.511639 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b88c3-2783-45fe-a2a1-00d00815b7db None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:05.512421 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b88c3-2783-45fe-a2a1-00d00815b7db None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:05.512916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b88c3-2783-45fe-a2a1-00d00815b7db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:05.514528 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b88c3-2783-45fe-a2a1-00d00815b7db None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:05.524268 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 114/231] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:32:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:05.530759 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc10b5-c464-466f-afa9-9a0743a3fb2a None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:05.562927 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:05.667649 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc10b5-c464-466f-afa9-9a0743a3fb2a None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:05.708562 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08657d72-d34b-4194-b1ff-88c53a55e503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:05.709480 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08657d72-d34b-4194-b1ff-88c53a55e503 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:05.709765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08657d72-d34b-4194-b1ff-88c53a55e503 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:05.710042 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08657d72-d34b-4194-b1ff-88c53a55e503 None admin] PATH_INFO: `/v3/users/79482c0d85544304a916213e9d65d108` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:05.711322 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08657d72-d34b-4194-b1ff-88c53a55e503 None admin] RBAC: Authorizing `identity:get_user(user_id=79482c0d85544304a916213e9d65d108)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:05.712838 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08657d72-d34b-4194-b1ff-88c53a55e503 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:05.714695 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 118/232] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:32:05 2022] GET /identity/v3/users/79482c0d85544304a916213e9d65d108 => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:05.720227 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b88c3-2783-45fe-a2a1-00d00815b7db None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:05.849264 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b88c3-2783-45fe-a2a1-00d00815b7db None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:05.850785 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:05.898428 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7de455-2f51-46a0-8c26-e0904e4cf0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:05.899311 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7de455-2f51-46a0-8c26-e0904e4cf0dc None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:05.899598 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7de455-2f51-46a0-8c26-e0904e4cf0dc None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:05.899878 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7de455-2f51-46a0-8c26-e0904e4cf0dc None admin] PATH_INFO: `/v3/projects/acd1ac1812eb4faeb54c342d511d42de` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:05.906204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7de455-2f51-46a0-8c26-e0904e4cf0dc None admin] RBAC: Authorizing `identity:get_project(project_id=acd1ac1812eb4faeb54c342d511d42de)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:05.908972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7de455-2f51-46a0-8c26-e0904e4cf0dc None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:05.912147 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 115/233] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:32:05 2022] GET /identity/v3/projects/acd1ac1812eb4faeb54c342d511d42de => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:05.920544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08657d72-d34b-4194-b1ff-88c53a55e503 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:06.035434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:06.058948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08657d72-d34b-4194-b1ff-88c53a55e503 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:06.096903 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851c8c8e-ce21-47a2-98f9-714f3d395b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:06.097853 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c8c8e-ce21-47a2-98f9-714f3d395b3f None admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:06.098138 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c8c8e-ce21-47a2-98f9-714f3d395b3f None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:06.098417 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c8c8e-ce21-47a2-98f9-714f3d395b3f None admin] PATH_INFO: `/v3/projects/acd1ac1812eb4faeb54c342d511d42de/users/79482c0d85544304a916213e9d65d108/roles/534400e7b28546a8a04c08f825c28566` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:06.100700 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851c8c8e-ce21-47a2-98f9-714f3d395b3f None admin] RBAC: Authorizing `identity:create_grant(project_id=acd1ac1812eb4faeb54c342d511d42de, user_id=79482c0d85544304a916213e9d65d108, role_id=534400e7b28546a8a04c08f825c28566)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:06.102110 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851c8c8e-ce21-47a2-98f9-714f3d395b3f None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:06.132112 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 119/234] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jul 6 13:32:05 2022] PUT /identity/v3/projects/acd1ac1812eb4faeb54c342d511d42de/users/79482c0d85544304a916213e9d65d108/roles/534400e7b28546a8a04c08f825c28566 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:32:06.841366 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80102c-7987-4330-8d92-96e9671a8d92 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:06.841917 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80102c-7987-4330-8d92-96e9671a8d92 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:06.841917 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80102c-7987-4330-8d92-96e9671a8d92 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:06.843289 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 116/235] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:06.848636 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb725bd-af5c-4d00-a522-2b44e540fc11 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:06.849511 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb725bd-af5c-4d00-a522-2b44e540fc11 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:06.849999 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb725bd-af5c-4d00-a522-2b44e540fc11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:06.874020 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-eeb725bd-af5c-4d00-a522-2b44e540fc11 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:06.875556 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-eeb725bd-af5c-4d00-a522-2b44e540fc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:06.950566 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04205a85-3b96-4e11-967c-d9f18848c150 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:06.950911 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04205a85-3b96-4e11-967c-d9f18848c150 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:06.951249 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04205a85-3b96-4e11-967c-d9f18848c150 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:06.952861 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 117/236] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:06.959742 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 120/237] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:06.960000 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4392912-ed7a-4fb4-b5ad-936fe9c07170 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:06.960471 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4392912-ed7a-4fb4-b5ad-936fe9c07170 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:06.960759 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4392912-ed7a-4fb4-b5ad-936fe9c07170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:06.993991 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-f4392912-ed7a-4fb4-b5ad-936fe9c07170 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:06.995188 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-f4392912-ed7a-4fb4-b5ad-936fe9c07170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:07.017705 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb725bd-af5c-4d00-a522-2b44e540fc11 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:07.115462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 118/238] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:07.150969 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:07.177298 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4392912-ed7a-4fb4-b5ad-936fe9c07170 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:07.193049 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb725bd-af5c-4d00-a522-2b44e540fc11 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:07.224110 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09380417-9f4a-4464-aa11-9feecbd36874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:07.224916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09380417-9f4a-4464-aa11-9feecbd36874 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:07.225229 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09380417-9f4a-4464-aa11-9feecbd36874 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:07.225508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09380417-9f4a-4464-aa11-9feecbd36874 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:07.226792 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09380417-9f4a-4464-aa11-9feecbd36874 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:07.228381 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09380417-9f4a-4464-aa11-9feecbd36874 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:07.230390 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 121/239] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:07.242199 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09380417-9f4a-4464-aa11-9feecbd36874 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:07.321044 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:07.324740 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4392912-ed7a-4fb4-b5ad-936fe9c07170 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:07.392450 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09380417-9f4a-4464-aa11-9feecbd36874 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:07.427547 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e4c01-5696-49b4-a1a5-fcdf09a22483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:07.428225 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e4c01-5696-49b4-a1a5-fcdf09a22483 None admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:07.428512 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e4c01-5696-49b4-a1a5-fcdf09a22483 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:07.428788 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e4c01-5696-49b4-a1a5-fcdf09a22483 None admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:07.429536 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e4c01-5696-49b4-a1a5-fcdf09a22483 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:07.431105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e4c01-5696-49b4-a1a5-fcdf09a22483 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:07.450537 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:07.451196 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05ed29-ef7a-4215-aa00-df59bbbe97fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:07.452016 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05ed29-ef7a-4215-aa00-df59bbbe97fb None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:07.452301 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05ed29-ef7a-4215-aa00-df59bbbe97fb None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:07.452580 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05ed29-ef7a-4215-aa00-df59bbbe97fb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:07.456134 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 122/240] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jul 6 13:32:07 2022] POST /identity/v3/users => generated 293 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:32:07.494220 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05ed29-ef7a-4215-aa00-df59bbbe97fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:07.507105 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-ef05ed29-ef7a-4215-aa00-df59bbbe97fb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 13:32:07.507105 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:07.507105 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:07.507105 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:07.507105 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:07.507105 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:07.507105 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:07.507105 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:07.507105 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:07.507105 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:07.507105 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:07.507105 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:07.507105 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:07.507105 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:07.507105 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:07.507105 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:07.508078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:07.508078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:07.508078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:07.508078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:07.508078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:07.508078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:07.508078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:07.508078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:07.508078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:07.508078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:07.508078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:07.508078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:07.508078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:07.508078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:07.508078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:07.508078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:07.508856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:07.508856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:07.508856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:07.508856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:07.508856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:07.508856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:07.508856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:07.508856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:07.508856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:07.508856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:07.508856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 13:32:07.508856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:07.508856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 119/241] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:32:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:32:07.515917 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e4c01-5696-49b4-a1a5-fcdf09a22483 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:07.534224 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be20699-2fdc-4d23-8f33-2d6c4834cf72 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:07.534517 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be20699-2fdc-4d23-8f33-2d6c4834cf72 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:07.534797 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be20699-2fdc-4d23-8f33-2d6c4834cf72 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:07.536271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 120/242] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:07.541909 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d014fa-bbd1-4d93-8c89-083d05c9a08d None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:07.542218 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d014fa-bbd1-4d93-8c89-083d05c9a08d None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:07.542473 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d014fa-bbd1-4d93-8c89-083d05c9a08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:07.574117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-86d014fa-bbd1-4d93-8c89-083d05c9a08d None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:07.575611 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-86d014fa-bbd1-4d93-8c89-083d05c9a08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:07.639548 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e4c01-5696-49b4-a1a5-fcdf09a22483 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:07.682455 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeea6543-0852-49d1-b187-f765d4a0eac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:07.683493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea6543-0852-49d1-b187-f765d4a0eac9 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:07.683989 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea6543-0852-49d1-b187-f765d4a0eac9 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:07.685096 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea6543-0852-49d1-b187-f765d4a0eac9 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:07.686418 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeea6543-0852-49d1-b187-f765d4a0eac9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:07.686927 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeea6543-0852-49d1-b187-f765d4a0eac9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:07.688645 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeea6543-0852-49d1-b187-f765d4a0eac9 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:07.699266 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 123/243] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jul 6 13:32:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:07.708395 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeea6543-0852-49d1-b187-f765d4a0eac9 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:07.739011 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 121/244] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:07.769566 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d014fa-bbd1-4d93-8c89-083d05c9a08d None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:07.790755 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:07.851018 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeea6543-0852-49d1-b187-f765d4a0eac9 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:07.895464 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caaee2d-97a9-4cf7-86c2-44001dbca5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:07.896376 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caaee2d-97a9-4cf7-86c2-44001dbca5c5 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:07.896665 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caaee2d-97a9-4cf7-86c2-44001dbca5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:07.896948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caaee2d-97a9-4cf7-86c2-44001dbca5c5 None admin] PATH_INFO: `/v3/users/122dbda8957f44d9a09145414cfb256e` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:07.906047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:07.917876 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caaee2d-97a9-4cf7-86c2-44001dbca5c5 None admin] RBAC: Authorizing `identity:get_user(user_id=122dbda8957f44d9a09145414cfb256e)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:07.920015 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caaee2d-97a9-4cf7-86c2-44001dbca5c5 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:07.922197 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 124/245] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:32:07 2022] GET /identity/v3/users/122dbda8957f44d9a09145414cfb256e => generated 297 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:07.923186 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d014fa-bbd1-4d93-8c89-083d05c9a08d None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:07.928933 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caaee2d-97a9-4cf7-86c2-44001dbca5c5 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:07.964177 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b20f37-395d-43d7-a683-e897bd377a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:07.965028 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b20f37-395d-43d7-a683-e897bd377a0f None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:07.965415 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b20f37-395d-43d7-a683-e897bd377a0f None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:07.966496 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b20f37-395d-43d7-a683-e897bd377a0f None admin] PATH_INFO: `/v3/roles/member` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:07.977146 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b20f37-395d-43d7-a683-e897bd377a0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:07.985970 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-b4b20f37-395d-43d7-a683-e897bd377a0f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:32:07.985970 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:07.985970 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:07.985970 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:07.985970 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:07.985970 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:07.985970 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:07.985970 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:07.985970 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:07.985970 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:07.985970 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:07.985970 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:07.985970 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:07.985970 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:07.985970 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:07.985970 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:07.986934 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:07.986934 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:07.986934 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:07.986934 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:07.986934 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:07.986934 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:07.986934 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:07.986934 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:07.986934 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:07.986934 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:07.986934 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:07.986934 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:07.986934 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:07.986934 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:07.986934 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:07.986934 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:07.988059 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:07.988059 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:07.988059 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:07.988059 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:07.988059 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:07.988059 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:07.988059 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:07.988059 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:07.988059 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:07.988059 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:07.988059 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:32:07.988059 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:07.988059 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 122/246] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:32:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:32:07.993750 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b20f37-395d-43d7-a683-e897bd377a0f None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:08.052876 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caaee2d-97a9-4cf7-86c2-44001dbca5c5 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:08.077387 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:08.094731 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62672f1-1800-49e3-8e52-8f47b10bb0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:08.095732 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62672f1-1800-49e3-8e52-8f47b10bb0d4 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:08.096217 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62672f1-1800-49e3-8e52-8f47b10bb0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:08.096781 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62672f1-1800-49e3-8e52-8f47b10bb0d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:08.106930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b20f37-395d-43d7-a683-e897bd377a0f None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:08.145052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762bfbc0-20c9-4221-b8b5-1774751f2179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:08.145800 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bfbc0-20c9-4221-b8b5-1774751f2179 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:08.146087 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bfbc0-20c9-4221-b8b5-1774751f2179 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:08.146367 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bfbc0-20c9-4221-b8b5-1774751f2179 None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:08.147246 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762bfbc0-20c9-4221-b8b5-1774751f2179 None admin] RBAC: Adding query filter params (name=member) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:08.147565 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762bfbc0-20c9-4221-b8b5-1774751f2179 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:08.149084 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762bfbc0-20c9-4221-b8b5-1774751f2179 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:08.159210 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 123/247] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jul 6 13:32:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:08.164738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762bfbc0-20c9-4221-b8b5-1774751f2179 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:08.177854 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62672f1-1800-49e3-8e52-8f47b10bb0d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:08.179761 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62672f1-1800-49e3-8e52-8f47b10bb0d4 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:08.181922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:08.284239 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762bfbc0-20c9-4221-b8b5-1774751f2179 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:08.288307 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 125/248] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:32:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:32:08.296629 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62672f1-1800-49e3-8e52-8f47b10bb0d4 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:08.320566 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:08.327741 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252dc31-3843-41f2-a4a1-858becdac92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:08.328958 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252dc31-3843-41f2-a4a1-858becdac92c None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:08.329515 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252dc31-3843-41f2-a4a1-858becdac92c None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:08.329982 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252dc31-3843-41f2-a4a1-858becdac92c None admin] PATH_INFO: `/v3/users/7ac97a3a7ed54185a876e936bf03bfb1` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:08.331388 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2252dc31-3843-41f2-a4a1-858becdac92c None admin] RBAC: Authorizing `identity:get_user(user_id=7ac97a3a7ed54185a876e936bf03bfb1)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:08.333285 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2252dc31-3843-41f2-a4a1-858becdac92c None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:08.335430 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 124/249] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:32:08 2022] GET /identity/v3/users/7ac97a3a7ed54185a876e936bf03bfb1 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:08.341989 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252dc31-3843-41f2-a4a1-858becdac92c None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:08.414852 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62672f1-1800-49e3-8e52-8f47b10bb0d4 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:08.451686 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:08.459184 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dded642-4793-40db-91b9-e56bc02e872d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:08.460336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dded642-4793-40db-91b9-e56bc02e872d None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:08.460905 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dded642-4793-40db-91b9-e56bc02e872d None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:08.461398 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dded642-4793-40db-91b9-e56bc02e872d None admin] PATH_INFO: `/v3/projects/acd1ac1812eb4faeb54c342d511d42de` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:08.462889 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dded642-4793-40db-91b9-e56bc02e872d None admin] RBAC: Authorizing `identity:get_project(project_id=acd1ac1812eb4faeb54c342d511d42de)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:08.464820 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dded642-4793-40db-91b9-e56bc02e872d None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:08.467341 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 126/250] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:32:08 2022] GET /identity/v3/projects/acd1ac1812eb4faeb54c342d511d42de => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:08.468760 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b512d-c122-4c45-9f20-2407a0c86128 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:08.469269 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b512d-c122-4c45-9f20-2407a0c86128 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:08.469814 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b512d-c122-4c45-9f20-2407a0c86128 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:08.471568 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 127/251] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:08.476951 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f5b14-48e3-4935-b037-7cbe28f96f69 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:08.478923 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252dc31-3843-41f2-a4a1-858becdac92c None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:08.479281 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f5b14-48e3-4935-b037-7cbe28f96f69 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:08.479634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f5b14-48e3-4935-b037-7cbe28f96f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:08.502641 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-3e4f5b14-48e3-4935-b037-7cbe28f96f69 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:08.504897 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-3e4f5b14-48e3-4935-b037-7cbe28f96f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:08.519313 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab001826-4e4b-48f9-8aa8-aa152796fbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:08.520594 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab001826-4e4b-48f9-8aa8-aa152796fbf9 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:08.520795 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab001826-4e4b-48f9-8aa8-aa152796fbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:08.521206 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab001826-4e4b-48f9-8aa8-aa152796fbf9 None admin] PATH_INFO: `/v3/projects/acd1ac1812eb4faeb54c342d511d42de` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:08.522564 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab001826-4e4b-48f9-8aa8-aa152796fbf9 None admin] RBAC: Authorizing `identity:get_project(project_id=acd1ac1812eb4faeb54c342d511d42de)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:08.524361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab001826-4e4b-48f9-8aa8-aa152796fbf9 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:08.526630 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 125/252] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:32:08 2022] GET /identity/v3/projects/acd1ac1812eb4faeb54c342d511d42de => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:08.527601 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab001826-4e4b-48f9-8aa8-aa152796fbf9 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:08.594217 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 128/253] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:08.595597 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4f5b14-48e3-4935-b037-7cbe28f96f69 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:08.619550 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:08.651304 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab001826-4e4b-48f9-8aa8-aa152796fbf9 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:08.695958 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76631c-f2e1-43ef-8ee8-0bec07e0b38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:08.697317 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76631c-f2e1-43ef-8ee8-0bec07e0b38a None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:08.697767 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76631c-f2e1-43ef-8ee8-0bec07e0b38a None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:08.698435 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76631c-f2e1-43ef-8ee8-0bec07e0b38a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:08.699413 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a76631c-f2e1-43ef-8ee8-0bec07e0b38a None admin] RBAC: Adding query filter params (role.id=65ecdcc5a9d0487ba529e54c626e152f, scope.project.id=acd1ac1812eb4faeb54c342d511d42de, user.id=122dbda8957f44d9a09145414cfb256e) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:08.700573 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a76631c-f2e1-43ef-8ee8-0bec07e0b38a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:08.702983 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a76631c-f2e1-43ef-8ee8-0bec07e0b38a None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:08.707135 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4f5b14-48e3-4935-b037-7cbe28f96f69 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:08.710899 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 126/254] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:32:08 2022] GET /identity/v3/role_assignments?user.id=122dbda8957f44d9a09145414cfb256e&scope.project.id=acd1ac1812eb4faeb54c342d511d42de&role.id=65ecdcc5a9d0487ba529e54c626e152f => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:08.711859 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76631c-f2e1-43ef-8ee8-0bec07e0b38a None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:08.743582 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:08.797744 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaff2f9-16de-466f-8859-2344763edfad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:08.799206 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaff2f9-16de-466f-8859-2344763edfad None admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:08.799619 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaff2f9-16de-466f-8859-2344763edfad None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:08.800022 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaff2f9-16de-466f-8859-2344763edfad None admin] PATH_INFO: `/v3/projects/acd1ac1812eb4faeb54c342d511d42de/users/7ac97a3a7ed54185a876e936bf03bfb1/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:08.802437 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaff2f9-16de-466f-8859-2344763edfad None admin] RBAC: Authorizing `identity:create_grant(project_id=acd1ac1812eb4faeb54c342d511d42de, user_id=7ac97a3a7ed54185a876e936bf03bfb1, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:08.804069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaff2f9-16de-466f-8859-2344763edfad None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:08.831255 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 129/255] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jul 6 13:32:08 2022] PUT /identity/v3/projects/acd1ac1812eb4faeb54c342d511d42de/users/7ac97a3a7ed54185a876e936bf03bfb1/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:32:08.832948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d40872-edc7-4ee2-b37b-f4d0280eb30f None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:08.833377 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d40872-edc7-4ee2-b37b-f4d0280eb30f None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:08.833812 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d40872-edc7-4ee2-b37b-f4d0280eb30f None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:08.835311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 130/256] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:08.841850 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5590e7-a7fd-4ec7-bd19-e4d27119ff3e None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:08.842881 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5590e7-a7fd-4ec7-bd19-e4d27119ff3e None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:08.842881 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5590e7-a7fd-4ec7-bd19-e4d27119ff3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:08.868813 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-5e5590e7-a7fd-4ec7-bd19-e4d27119ff3e None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:08.870192 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-5e5590e7-a7fd-4ec7-bd19-e4d27119ff3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:08.877689 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76631c-f2e1-43ef-8ee8-0bec07e0b38a None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:08.948483 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c252c49c-7869-491c-8392-294d5b94fa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:08.949382 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252c49c-7869-491c-8392-294d5b94fa55 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:08.949671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252c49c-7869-491c-8392-294d5b94fa55 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:08.949978 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252c49c-7869-491c-8392-294d5b94fa55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:08.962508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c252c49c-7869-491c-8392-294d5b94fa55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:08.971209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-c252c49c-7869-491c-8392-294d5b94fa55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:32:08.971209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:08.971209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:08.971209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:08.971209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:08.971209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:08.971209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:08.971209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:08.971209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:08.971209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:08.971209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:08.971209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:08.971209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:08.971209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:08.971209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:08.971209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:08.972180 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:08.972180 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:08.972180 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:08.972180 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:08.972180 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:08.972180 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:08.972180 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:08.972180 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:08.972180 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:08.972180 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:08.972180 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:08.972180 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:08.972180 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:08.972180 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:08.972180 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:08.972180 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:08.972966 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:08.972966 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:08.972966 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:08.972966 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:08.972966 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:08.972966 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:08.972966 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:08.972966 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:08.972966 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:08.972966 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:08.972966 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:32:08.972966 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:08.972966 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 127/257] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:32:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:08.980034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c252c49c-7869-491c-8392-294d5b94fa55 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:09.011081 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 131/258] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:09.062091 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5590e7-a7fd-4ec7-bd19-e4d27119ff3e None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:09.085400 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:09.133490 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:09.140640 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c252c49c-7869-491c-8392-294d5b94fa55 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:09.208422 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51507ef3-12a7-4b73-ba46-56b78ab935ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:09.209118 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51507ef3-12a7-4b73-ba46-56b78ab935ac None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:09.209402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51507ef3-12a7-4b73-ba46-56b78ab935ac None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:09.209680 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51507ef3-12a7-4b73-ba46-56b78ab935ac None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:09.210444 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51507ef3-12a7-4b73-ba46-56b78ab935ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:09.210751 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51507ef3-12a7-4b73-ba46-56b78ab935ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:09.212126 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51507ef3-12a7-4b73-ba46-56b78ab935ac None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:09.231877 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 128/259] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:32:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:09.237581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51507ef3-12a7-4b73-ba46-56b78ab935ac None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:09.256170 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5590e7-a7fd-4ec7-bd19-e4d27119ff3e None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:09.353745 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424b884-46ae-4115-959f-758f78ec7dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:09.354591 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424b884-46ae-4115-959f-758f78ec7dcc None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:09.354881 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424b884-46ae-4115-959f-758f78ec7dcc None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:09.355163 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424b884-46ae-4115-959f-758f78ec7dcc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:09.384346 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51507ef3-12a7-4b73-ba46-56b78ab935ac None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:09.409026 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9424b884-46ae-4115-959f-758f78ec7dcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:09.422275 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-9424b884-46ae-4115-959f-758f78ec7dcc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 13:32:09.422275 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:09.422275 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:09.422275 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:09.422275 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:09.422275 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:09.422275 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:09.422275 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:09.422275 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:09.422275 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:09.422275 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:09.422275 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:09.422275 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:09.422275 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:09.422275 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:09.422275 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:09.423156 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:09.423156 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:09.423156 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:09.423156 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:09.423156 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:09.423156 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:09.423156 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:09.423156 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:09.423156 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:09.423156 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:09.423156 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:09.423156 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:09.423156 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:09.423156 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:09.423156 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:09.423156 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:09.424630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:09.424630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:09.424630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:09.424630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:09.424630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:09.424630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:09.424630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:09.424630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:09.424630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:09.424630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:09.424630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 13:32:09.424630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:09.424630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 132/260] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jul 6 13:32:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:09.429610 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424b884-46ae-4115-959f-758f78ec7dcc None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:09.467221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008a7ba-8d6d-4d08-99f2-de3f5cde1a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:09.468472 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008a7ba-8d6d-4d08-99f2-de3f5cde1a52 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:09.468960 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008a7ba-8d6d-4d08-99f2-de3f5cde1a52 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:09.469457 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008a7ba-8d6d-4d08-99f2-de3f5cde1a52 None admin] PATH_INFO: `/v3/users/79482c0d85544304a916213e9d65d108` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:09.472718 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5008a7ba-8d6d-4d08-99f2-de3f5cde1a52 None admin] RBAC: Authorizing `identity:get_user(user_id=79482c0d85544304a916213e9d65d108)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:09.474450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5008a7ba-8d6d-4d08-99f2-de3f5cde1a52 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:09.476958 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 129/261] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:32:09 2022] GET /identity/v3/users/79482c0d85544304a916213e9d65d108 => generated 283 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:09.483169 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008a7ba-8d6d-4d08-99f2-de3f5cde1a52 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:09.497331 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:09.561182 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:09.610516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424b884-46ae-4115-959f-758f78ec7dcc None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:09.670682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f304aae-cea8-48fd-b24b-45369317cb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:09.671296 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f304aae-cea8-48fd-b24b-45369317cb97 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:09.671582 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f304aae-cea8-48fd-b24b-45369317cb97 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:09.671858 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f304aae-cea8-48fd-b24b-45369317cb97 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:09.672632 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f304aae-cea8-48fd-b24b-45369317cb97 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:09.672933 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f304aae-cea8-48fd-b24b-45369317cb97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:09.674616 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f304aae-cea8-48fd-b24b-45369317cb97 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:09.695697 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 133/262] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:32:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:09.701527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f304aae-cea8-48fd-b24b-45369317cb97 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:09.704666 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008a7ba-8d6d-4d08-99f2-de3f5cde1a52 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:09.760617 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb5645-5f94-414d-a6eb-e55b722379c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:09.761316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb5645-5f94-414d-a6eb-e55b722379c6 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:09.761600 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb5645-5f94-414d-a6eb-e55b722379c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:09.761877 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb5645-5f94-414d-a6eb-e55b722379c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:09.938420 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdb5645-5f94-414d-a6eb-e55b722379c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:09.940064 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdb5645-5f94-414d-a6eb-e55b722379c6 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:09.942581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:09.945971 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f304aae-cea8-48fd-b24b-45369317cb97 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:10.006626 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:10.024467 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a990da4-6800-4a46-8fe3-de7424b62560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:10.025813 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a990da4-6800-4a46-8fe3-de7424b62560 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:10.026307 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a990da4-6800-4a46-8fe3-de7424b62560 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:10.026787 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a990da4-6800-4a46-8fe3-de7424b62560 None admin] PATH_INFO: `/v3/users/79482c0d85544304a916213e9d65d108` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:10.032142 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a990da4-6800-4a46-8fe3-de7424b62560 None admin] RBAC: Authorizing `identity:get_user(user_id=79482c0d85544304a916213e9d65d108)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:10.033917 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a990da4-6800-4a46-8fe3-de7424b62560 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:10.036090 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 134/263] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:32:09 2022] GET /identity/v3/users/79482c0d85544304a916213e9d65d108 => generated 283 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:10.042311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a990da4-6800-4a46-8fe3-de7424b62560 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:10.139542 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 130/264] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:32:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 657 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:32:10.146026 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb5645-5f94-414d-a6eb-e55b722379c6 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:10.276781 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a990da4-6800-4a46-8fe3-de7424b62560 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:10.348266 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb5645-5f94-414d-a6eb-e55b722379c6 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:10.350364 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:10.361782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12877c5b-e2cb-439c-a5d8-ff5899de3e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:10.362623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12877c5b-e2cb-439c-a5d8-ff5899de3e63 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:10.362915 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12877c5b-e2cb-439c-a5d8-ff5899de3e63 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:10.363193 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12877c5b-e2cb-439c-a5d8-ff5899de3e63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:10.417748 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9317d3-be98-4c0e-b94d-ce2a8a07502f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:10.419656 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9317d3-be98-4c0e-b94d-ce2a8a07502f None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:10.420145 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9317d3-be98-4c0e-b94d-ce2a8a07502f None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:10.420621 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9317d3-be98-4c0e-b94d-ce2a8a07502f None admin] PATH_INFO: `/v3/projects/acd1ac1812eb4faeb54c342d511d42de` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:10.422158 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9317d3-be98-4c0e-b94d-ce2a8a07502f None admin] RBAC: Authorizing `identity:get_project(project_id=acd1ac1812eb4faeb54c342d511d42de)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:10.423834 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9317d3-be98-4c0e-b94d-ce2a8a07502f None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:10.426032 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 131/265] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:32:10 2022] GET /identity/v3/projects/acd1ac1812eb4faeb54c342d511d42de => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:10.432022 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9317d3-be98-4c0e-b94d-ce2a8a07502f None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:10.555066 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:10.561848 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12877c5b-e2cb-439c-a5d8-ff5899de3e63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:10.563525 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12877c5b-e2cb-439c-a5d8-ff5899de3e63 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:10.618671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9317d3-be98-4c0e-b94d-ce2a8a07502f None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:10.707329 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174aaaf1-5ef8-4fe7-a904-7c51d3e8e193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:10.708120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174aaaf1-5ef8-4fe7-a904-7c51d3e8e193 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:10.708410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174aaaf1-5ef8-4fe7-a904-7c51d3e8e193 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:10.708685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174aaaf1-5ef8-4fe7-a904-7c51d3e8e193 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:10.709837 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174aaaf1-5ef8-4fe7-a904-7c51d3e8e193 None admin] RBAC: Adding query filter params (role.id=795daa2c78354fd48e4a741e1182cc98, scope.project.id=acd1ac1812eb4faeb54c342d511d42de, user.id=79482c0d85544304a916213e9d65d108) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:10.710154 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174aaaf1-5ef8-4fe7-a904-7c51d3e8e193 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:10.711595 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174aaaf1-5ef8-4fe7-a904-7c51d3e8e193 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:10.726277 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 132/266] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:32:10 2022] GET /identity/v3/role_assignments?user.id=79482c0d85544304a916213e9d65d108&scope.project.id=acd1ac1812eb4faeb54c342d511d42de&role.id=795daa2c78354fd48e4a741e1182cc98 => generated 608 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:10.749186 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 135/267] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:32:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 708 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:32:10.754710 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174aaaf1-5ef8-4fe7-a904-7c51d3e8e193 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:10.813320 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:10.915505 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174aaaf1-5ef8-4fe7-a904-7c51d3e8e193 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:10.964340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f79e8-6ce2-4a13-b7ba-6dd69357279a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:10.965253 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f79e8-6ce2-4a13-b7ba-6dd69357279a None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:10.965600 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f79e8-6ce2-4a13-b7ba-6dd69357279a None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:10.966442 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f79e8-6ce2-4a13-b7ba-6dd69357279a None admin] PATH_INFO: `/v3/projects/acd1ac1812eb4faeb54c342d511d42de` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:10.973757 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632f79e8-6ce2-4a13-b7ba-6dd69357279a None admin] RBAC: Authorizing `identity:get_project(project_id=acd1ac1812eb4faeb54c342d511d42de)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:10.975257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632f79e8-6ce2-4a13-b7ba-6dd69357279a None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:10.977020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 133/268] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:32:10 2022] GET /identity/v3/projects/acd1ac1812eb4faeb54c342d511d42de => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:10.984074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12877c5b-e2cb-439c-a5d8-ff5899de3e63 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:11.095163 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55977cd0-d012-4e37-a9ae-e0eb5021efc3 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:11.095450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55977cd0-d012-4e37-a9ae-e0eb5021efc3 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:11.095654 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55977cd0-d012-4e37-a9ae-e0eb5021efc3 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:11.097047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 134/269] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:32:11.103823 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d67e7-1571-4d06-919f-0257eb4e0a54 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:11.104115 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d67e7-1571-4d06-919f-0257eb4e0a54 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:11.104398 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d67e7-1571-4d06-919f-0257eb4e0a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:11.132035 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-4a9d67e7-1571-4d06-919f-0257eb4e0a54 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:11.135663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12877c5b-e2cb-439c-a5d8-ff5899de3e63 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:11.137279 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-4a9d67e7-1571-4d06-919f-0257eb4e0a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:11.137440 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:11.201337 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9069d7-f578-4117-81df-fffa499132a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:11.202014 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9069d7-f578-4117-81df-fffa499132a7 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:11.202295 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9069d7-f578-4117-81df-fffa499132a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:11.202568 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9069d7-f578-4117-81df-fffa499132a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:11.203415 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9069d7-f578-4117-81df-fffa499132a7 None admin] RBAC: Adding query filter params (role.id=534400e7b28546a8a04c08f825c28566, scope.project.id=acd1ac1812eb4faeb54c342d511d42de, user.id=79482c0d85544304a916213e9d65d108) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:11.203725 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9069d7-f578-4117-81df-fffa499132a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:11.205206 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9069d7-f578-4117-81df-fffa499132a7 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:11.215170 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 136/270] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:32:10 2022] GET /identity/v3/role_assignments?user.id=79482c0d85544304a916213e9d65d108&scope.project.id=acd1ac1812eb4faeb54c342d511d42de&role.id=534400e7b28546a8a04c08f825c28566 => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:11.216670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7fceb-4651-477d-a369-69f545758bd0 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:11.216960 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7fceb-4651-477d-a369-69f545758bd0 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:11.217266 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7fceb-4651-477d-a369-69f545758bd0 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:11.219847 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 137/271] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:11.226554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59297398-349a-4717-be88-2f7c85550123 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:11.226863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59297398-349a-4717-be88-2f7c85550123 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:11.227111 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59297398-349a-4717-be88-2f7c85550123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:11.257719 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-59297398-349a-4717-be88-2f7c85550123 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:11.258951 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-59297398-349a-4717-be88-2f7c85550123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:11.266247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 135/272] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 06 13:32:11.310790 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d67e7-1571-4d06-919f-0257eb4e0a54 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:11.379563 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 138/273] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:11.379873 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:11.423480 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59297398-349a-4717-be88-2f7c85550123 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:11.485544 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d67e7-1571-4d06-919f-0257eb4e0a54 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:11.527250 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67cd90-4eec-4478-9424-a8482f567e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:11.529348 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67cd90-4eec-4478-9424-a8482f567e0d None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:11.529843 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67cd90-4eec-4478-9424-a8482f567e0d None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:11.530374 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67cd90-4eec-4478-9424-a8482f567e0d None admin] PATH_INFO: `/v3/roles/member` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:11.541905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a67cd90-4eec-4478-9424-a8482f567e0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:11.551848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-8a67cd90-4eec-4478-9424-a8482f567e0d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:32:11.551848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:11.551848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:11.551848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:11.551848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:11.551848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:11.551848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:11.551848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:11.551848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:11.551848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:11.551848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:11.551848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:11.551848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:11.551848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:11.551848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:11.551848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:11.552774 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:11.552774 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:11.552774 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:11.552774 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:11.552774 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:11.552774 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:11.552774 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:11.552774 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:11.552774 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:11.552774 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:11.552774 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:11.552774 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:11.552774 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:11.552774 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:11.552774 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:11.552774 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:11.553922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:11.553922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:11.553922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:11.553922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:11.553922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:11.553922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:11.553922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:11.553922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:11.553922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:11.553922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:11.553922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:32:11.553922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:11.556470 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 136/274] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:32:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:32:11.557417 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:11.562910 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67cd90-4eec-4478-9424-a8482f567e0d None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:11.572574 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59297398-349a-4717-be88-2f7c85550123 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:11.617735 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d85162-2dff-40a9-aadc-3171ef9a58e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:11.619052 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d85162-2dff-40a9-aadc-3171ef9a58e7 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:11.619588 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d85162-2dff-40a9-aadc-3171ef9a58e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:11.620087 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d85162-2dff-40a9-aadc-3171ef9a58e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:11.623764 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d85162-2dff-40a9-aadc-3171ef9a58e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:11.625602 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d85162-2dff-40a9-aadc-3171ef9a58e7 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:11.628041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 139/275] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:11.633581 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d85162-2dff-40a9-aadc-3171ef9a58e7 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:11.692242 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:11.707039 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67cd90-4eec-4478-9424-a8482f567e0d None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:11.744218 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdaec1b-0aa4-4993-8d96-f896345a8710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:11.744882 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdaec1b-0aa4-4993-8d96-f896345a8710 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:11.745191 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdaec1b-0aa4-4993-8d96-f896345a8710 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:11.745464 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdaec1b-0aa4-4993-8d96-f896345a8710 None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:11.746210 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdaec1b-0aa4-4993-8d96-f896345a8710 None admin] RBAC: Adding query filter params (name=member) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:11.746519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdaec1b-0aa4-4993-8d96-f896345a8710 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:11.747932 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdaec1b-0aa4-4993-8d96-f896345a8710 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:11.756638 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 137/276] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jul 6 13:32:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:11.757720 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5584a-512b-4f3e-baa5-ae29f187c958 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:11.758006 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5584a-512b-4f3e-baa5-ae29f187c958 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:11.758279 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5584a-512b-4f3e-baa5-ae29f187c958 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:11.759494 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 138/277] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:11.762148 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf5584a-512b-4f3e-baa5-ae29f187c958 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:11.769476 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d85162-2dff-40a9-aadc-3171ef9a58e7 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:11.806765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa722a41-edf6-4db6-83d9-d3d703fd5fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:11.807471 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa722a41-edf6-4db6-83d9-d3d703fd5fcd None admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:11.807754 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa722a41-edf6-4db6-83d9-d3d703fd5fcd None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:11.808032 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa722a41-edf6-4db6-83d9-d3d703fd5fcd None admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:11.808760 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa722a41-edf6-4db6-83d9-d3d703fd5fcd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:11.810564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa722a41-edf6-4db6-83d9-d3d703fd5fcd None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:11.830015 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 140/278] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jul 6 13:32:11 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:32:11.831503 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce04f01-279f-4675-83b4-058100c23718 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:11.831953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce04f01-279f-4675-83b4-058100c23718 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:11.832237 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce04f01-279f-4675-83b4-058100c23718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:11.854860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-2ce04f01-279f-4675-83b4-058100c23718 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:11.856395 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-2ce04f01-279f-4675-83b4-058100c23718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:11.870208 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:11.884118 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf5584a-512b-4f3e-baa5-ae29f187c958 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:11.926637 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb2cb06-21d5-4ad8-8278-ffe2401219ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:11.927499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2cb06-21d5-4ad8-8278-ffe2401219ab None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:11.927807 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2cb06-21d5-4ad8-8278-ffe2401219ab None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:11.928190 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2cb06-21d5-4ad8-8278-ffe2401219ab None admin] PATH_INFO: `/v3/users/7b713d8aedf846c7887f734bb55736b8` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:11.941043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 141/279] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:11.941043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e7acd-b5c9-4b30-b7ce-542a2cb018f5 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:11.941043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e7acd-b5c9-4b30-b7ce-542a2cb018f5 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:11.941043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e7acd-b5c9-4b30-b7ce-542a2cb018f5 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:11.941043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 142/280] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:11.951744 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f03a9-f4c0-443f-bbea-79c053c7f447 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:11.952032 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f03a9-f4c0-443f-bbea-79c053c7f447 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:11.952323 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f03a9-f4c0-443f-bbea-79c053c7f447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:11.956415 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb2cb06-21d5-4ad8-8278-ffe2401219ab None admin] RBAC: Authorizing `identity:get_user(user_id=7b713d8aedf846c7887f734bb55736b8)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:11.959120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb2cb06-21d5-4ad8-8278-ffe2401219ab None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:11.960964 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 139/281] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:32:11 2022] GET /identity/v3/users/7b713d8aedf846c7887f734bb55736b8 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:11.964661 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb2cb06-21d5-4ad8-8278-ffe2401219ab None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:11.973124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-a54f03a9-f4c0-443f-bbea-79c053c7f447 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:11.974596 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-a54f03a9-f4c0-443f-bbea-79c053c7f447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:11.983390 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:12.057619 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 143/282] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 13:32:12.066229 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54f03a9-f4c0-443f-bbea-79c053c7f447 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:12.098399 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb2cb06-21d5-4ad8-8278-ffe2401219ab None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:12.140476 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b04e61e-52b9-4c22-855f-c1e212852a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:12.141590 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04e61e-52b9-4c22-855f-c1e212852a24 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:12.141884 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04e61e-52b9-4c22-855f-c1e212852a24 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:12.142163 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04e61e-52b9-4c22-855f-c1e212852a24 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:12.155269 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b04e61e-52b9-4c22-855f-c1e212852a24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:12.165099 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-4b04e61e-52b9-4c22-855f-c1e212852a24 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 13:32:12.165099 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:12.165099 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:12.165099 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:12.165099 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:12.165099 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:12.165099 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:12.165099 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:12.165099 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:12.165099 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:12.165099 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:12.165099 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:12.165099 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:12.165099 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:12.165099 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:12.165099 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:12.166802 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:12.166802 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:12.166802 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:12.166802 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:12.166802 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:12.166802 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:12.166802 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:12.166802 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:12.166802 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:12.166802 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:12.166802 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:12.166802 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:12.166802 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:12.166802 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:12.166802 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:12.166802 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:12.167629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:12.167629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:12.167629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:12.167629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:12.167629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:12.167629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:12.167629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:12.167629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:12.167629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:12.167629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:12.167629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 13:32:12.167629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:12.167629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 140/283] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:32:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:32:12.168295 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:12.168364 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b04e61e-52b9-4c22-855f-c1e212852a24 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:12.202457 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54f03a9-f4c0-443f-bbea-79c053c7f447 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:12.249940 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b501893-f638-4e21-aaca-f1a1482355a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:12.250600 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b501893-f638-4e21-aaca-f1a1482355a6 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:12.250879 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b501893-f638-4e21-aaca-f1a1482355a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:12.251153 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b501893-f638-4e21-aaca-f1a1482355a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:12.252242 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b501893-f638-4e21-aaca-f1a1482355a6 None admin] RBAC: Adding query filter params (role.id=b3244d1d9f004c1b8d3594fd30da1024, user.id=7b713d8aedf846c7887f734bb55736b8, scope.system=all) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:12.252552 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b501893-f638-4e21-aaca-f1a1482355a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:12.254624 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b501893-f638-4e21-aaca-f1a1482355a6 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:12.258402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:12.264868 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 144/284] 10.222.0.20 () {56 vars in 1308 bytes} [Wed Jul 6 13:32:12 2022] GET /identity/v3/role_assignments?user.id=7b713d8aedf846c7887f734bb55736b8&scope.system=all&role.id=b3244d1d9f004c1b8d3594fd30da1024 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:12.265741 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b501893-f638-4e21-aaca-f1a1482355a6 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:12.357291 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b04e61e-52b9-4c22-855f-c1e212852a24 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:12.409319 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b501893-f638-4e21-aaca-f1a1482355a6 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:12.447772 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad372af-801b-40e5-94c7-a8f95ac7d3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:12.448837 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad372af-801b-40e5-94c7-a8f95ac7d3d7 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:12.449152 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad372af-801b-40e5-94c7-a8f95ac7d3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:12.449428 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad372af-801b-40e5-94c7-a8f95ac7d3d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:12.461806 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad372af-801b-40e5-94c7-a8f95ac7d3d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:12.466233 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383d9f5-923e-4ab8-a996-0e42c89dd0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:12.467168 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383d9f5-923e-4ab8-a996-0e42c89dd0c4 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:12.467650 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383d9f5-923e-4ab8-a996-0e42c89dd0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:12.468144 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383d9f5-923e-4ab8-a996-0e42c89dd0c4 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:12.469127 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d383d9f5-923e-4ab8-a996-0e42c89dd0c4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:12.469635 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d383d9f5-923e-4ab8-a996-0e42c89dd0c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:12.470329 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-aad372af-801b-40e5-94c7-a8f95ac7d3d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:32:12.470329 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:12.470329 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:12.470329 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:12.470329 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:12.470329 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:12.470329 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:12.470329 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:12.470329 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:12.470329 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:12.470329 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:12.470329 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:12.470329 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:12.470329 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:12.470329 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:12.470329 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:12.471223 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:12.471223 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:12.471223 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:12.471223 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:12.471223 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:12.471223 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:12.471223 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:12.471223 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:12.471223 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:12.471223 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:12.471223 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:12.471223 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:12.471223 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:12.471223 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:12.471223 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:12.471223 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:12.476813 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:12.476813 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:12.476813 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:12.476813 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:12.476813 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:12.476813 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:12.476813 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:12.476813 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:12.476813 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:12.476813 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:12.476813 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:32:12.476813 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:12.476813 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 141/285] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:32:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:12.477492 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad372af-801b-40e5-94c7-a8f95ac7d3d7 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:12.478815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d383d9f5-923e-4ab8-a996-0e42c89dd0c4 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:12.496311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 145/286] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jul 6 13:32:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:12.509936 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383d9f5-923e-4ab8-a996-0e42c89dd0c4 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:12.515930 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:12.595808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad372af-801b-40e5-94c7-a8f95ac7d3d7 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:12.631286 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc0d62-71fb-4491-944d-167de20c89a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:12.632303 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc0d62-71fb-4491-944d-167de20c89a3 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:12.632809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc0d62-71fb-4491-944d-167de20c89a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:12.633329 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc0d62-71fb-4491-944d-167de20c89a3 None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:12.634510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc0d62-71fb-4491-944d-167de20c89a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:12.635027 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc0d62-71fb-4491-944d-167de20c89a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:12.636595 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc0d62-71fb-4491-944d-167de20c89a3 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:12.646405 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 142/287] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jul 6 13:32:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:12.651116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:12.652226 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc0d62-71fb-4491-944d-167de20c89a3 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:12.692530 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383d9f5-923e-4ab8-a996-0e42c89dd0c4 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:12.763031 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cad21d-33c8-49d0-8f2b-37bdada101b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:12.764306 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cad21d-33c8-49d0-8f2b-37bdada101b1 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:12.764810 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cad21d-33c8-49d0-8f2b-37bdada101b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:12.765297 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cad21d-33c8-49d0-8f2b-37bdada101b1 None admin] PATH_INFO: `/v3/users/122dbda8957f44d9a09145414cfb256e` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:12.768028 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cad21d-33c8-49d0-8f2b-37bdada101b1 None admin] RBAC: Authorizing `identity:get_user(user_id=122dbda8957f44d9a09145414cfb256e)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:12.769724 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cad21d-33c8-49d0-8f2b-37bdada101b1 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:12.773704 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc0d62-71fb-4491-944d-167de20c89a3 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:12.775760 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 146/288] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:32:12 2022] GET /identity/v3/users/122dbda8957f44d9a09145414cfb256e => generated 297 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:12.797401 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cad21d-33c8-49d0-8f2b-37bdada101b1 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:12.831731 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467338fe-843b-4a54-ae7f-b8770d8503e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:12.833092 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467338fe-843b-4a54-ae7f-b8770d8503e5 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:12.833594 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467338fe-843b-4a54-ae7f-b8770d8503e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:12.834060 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467338fe-843b-4a54-ae7f-b8770d8503e5 None admin] PATH_INFO: `/v3/users/7ac97a3a7ed54185a876e936bf03bfb1` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:12.835681 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467338fe-843b-4a54-ae7f-b8770d8503e5 None admin] RBAC: Authorizing `identity:get_user(user_id=7ac97a3a7ed54185a876e936bf03bfb1)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:12.837407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467338fe-843b-4a54-ae7f-b8770d8503e5 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:12.839549 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 143/289] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:32:12 2022] GET /identity/v3/users/7ac97a3a7ed54185a876e936bf03bfb1 => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:12.845625 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467338fe-843b-4a54-ae7f-b8770d8503e5 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:12.909515 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:12.950556 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467338fe-843b-4a54-ae7f-b8770d8503e5 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:12.975518 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:12.997289 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e7140-c1d7-4209-9226-2f730c5be8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:12.998543 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e7140-c1d7-4209-9226-2f730c5be8e9 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:12.998822 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e7140-c1d7-4209-9226-2f730c5be8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:12.999106 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e7140-c1d7-4209-9226-2f730c5be8e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:13.028671 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cad21d-33c8-49d0-8f2b-37bdada101b1 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:13.080709 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b9d7f-e6d5-4f34-b7c0-e74dbc05c470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:13.081707 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b9d7f-e6d5-4f34-b7c0-e74dbc05c470 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:13.081963 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b9d7f-e6d5-4f34-b7c0-e74dbc05c470 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:13.082245 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b9d7f-e6d5-4f34-b7c0-e74dbc05c470 None admin] PATH_INFO: `/v3/projects/acd1ac1812eb4faeb54c342d511d42de` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:13.083460 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b9d7f-e6d5-4f34-b7c0-e74dbc05c470 None admin] RBAC: Authorizing `identity:get_project(project_id=acd1ac1812eb4faeb54c342d511d42de)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:13.084931 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b9d7f-e6d5-4f34-b7c0-e74dbc05c470 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:13.087044 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 147/290] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:32:12 2022] GET /identity/v3/projects/acd1ac1812eb4faeb54c342d511d42de => generated 302 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:13.093612 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b9d7f-e6d5-4f34-b7c0-e74dbc05c470 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:13.126176 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e7140-c1d7-4209-9226-2f730c5be8e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:13.127657 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e7140-c1d7-4209-9226-2f730c5be8e9 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:13.196317 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:13.236883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 144/291] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:32:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 06 13:32:13.241950 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e7140-c1d7-4209-9226-2f730c5be8e9 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:13.344291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b9d7f-e6d5-4f34-b7c0-e74dbc05c470 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:13.411230 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e7140-c1d7-4209-9226-2f730c5be8e9 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:13.447714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677870d3-8636-4d6a-a096-27df472daa9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:13.448617 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677870d3-8636-4d6a-a096-27df472daa9f None admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:13.448902 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677870d3-8636-4d6a-a096-27df472daa9f None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:13.449200 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677870d3-8636-4d6a-a096-27df472daa9f None admin] PATH_INFO: `/v3/projects/acd1ac1812eb4faeb54c342d511d42de/users/122dbda8957f44d9a09145414cfb256e/roles/65ecdcc5a9d0487ba529e54c626e152f` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:13.457322 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677870d3-8636-4d6a-a096-27df472daa9f None admin] RBAC: Authorizing `identity:create_grant(project_id=acd1ac1812eb4faeb54c342d511d42de, user_id=122dbda8957f44d9a09145414cfb256e, role_id=65ecdcc5a9d0487ba529e54c626e152f)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:13.458750 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677870d3-8636-4d6a-a096-27df472daa9f None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:13.465140 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e871a4-f0a4-425b-b971-0d84da0b0ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:13.467988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e871a4-f0a4-425b-b971-0d84da0b0ff8 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:13.468275 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e871a4-f0a4-425b-b971-0d84da0b0ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:13.468550 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e871a4-f0a4-425b-b971-0d84da0b0ff8 None admin] PATH_INFO: `/v3/projects/acd1ac1812eb4faeb54c342d511d42de` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:13.470918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e871a4-f0a4-425b-b971-0d84da0b0ff8 None admin] RBAC: Authorizing `identity:get_project(project_id=acd1ac1812eb4faeb54c342d511d42de)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:13.472504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e871a4-f0a4-425b-b971-0d84da0b0ff8 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:13.474396 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 145/292] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:32:13 2022] GET /identity/v3/projects/acd1ac1812eb4faeb54c342d511d42de => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:13.480381 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e871a4-f0a4-425b-b971-0d84da0b0ff8 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:13.484809 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:13.489933 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 148/293] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jul 6 13:32:13 2022] PUT /identity/v3/projects/acd1ac1812eb4faeb54c342d511d42de/users/122dbda8957f44d9a09145414cfb256e/roles/65ecdcc5a9d0487ba529e54c626e152f => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:32:13.537414 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:13.645220 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e871a4-f0a4-425b-b971-0d84da0b0ff8 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:13.688415 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1b398-6584-408f-b490-514e3e6ce97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:13.689112 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1b398-6584-408f-b490-514e3e6ce97c None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:13.689395 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1b398-6584-408f-b490-514e3e6ce97c None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:13.689670 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1b398-6584-408f-b490-514e3e6ce97c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:13.690510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1b398-6584-408f-b490-514e3e6ce97c None admin] RBAC: Adding query filter params (role.id=b3244d1d9f004c1b8d3594fd30da1024, scope.project.id=acd1ac1812eb4faeb54c342d511d42de, user.id=7ac97a3a7ed54185a876e936bf03bfb1) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:13.690819 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1b398-6584-408f-b490-514e3e6ce97c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:13.692242 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1b398-6584-408f-b490-514e3e6ce97c None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:13.698810 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 146/294] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:32:13 2022] GET /identity/v3/role_assignments?user.id=7ac97a3a7ed54185a876e936bf03bfb1&scope.project.id=acd1ac1812eb4faeb54c342d511d42de&role.id=b3244d1d9f004c1b8d3594fd30da1024 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:14.299330 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187de8ed-8d04-44db-a660-b72d3bca11f4 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:14.300164 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187de8ed-8d04-44db-a660-b72d3bca11f4 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:14.300647 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187de8ed-8d04-44db-a660-b72d3bca11f4 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:14.302290 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 149/295] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:14.307925 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e19c50-e980-4e00-9ce2-5c512da4896b None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:14.308475 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e19c50-e980-4e00-9ce2-5c512da4896b None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:14.308947 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e19c50-e980-4e00-9ce2-5c512da4896b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:14.329086 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-b4e19c50-e980-4e00-9ce2-5c512da4896b None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:14.330592 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-b4e19c50-e980-4e00-9ce2-5c512da4896b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:14.385020 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dff84e-a6b8-42b3-86c5-901d58450538 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:14.385347 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dff84e-a6b8-42b3-86c5-901d58450538 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:14.385620 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dff84e-a6b8-42b3-86c5-901d58450538 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:14.387152 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 150/296] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:14.392213 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05752bd-720c-4e70-9e94-f46cf5034b59 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:14.392501 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05752bd-720c-4e70-9e94-f46cf5034b59 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:14.392782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05752bd-720c-4e70-9e94-f46cf5034b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:14.414149 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 147/297] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:14.415939 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-f05752bd-720c-4e70-9e94-f46cf5034b59 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:14.421810 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-f05752bd-720c-4e70-9e94-f46cf5034b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:14.439379 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e19c50-e980-4e00-9ce2-5c512da4896b None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:14.466702 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:14.509022 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 151/298] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 13:32:14.528302 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05752bd-720c-4e70-9e94-f46cf5034b59 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:14.556588 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e19c50-e980-4e00-9ce2-5c512da4896b None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:14.595419 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6134f163-6b60-458a-a53c-d1e6f2f36769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:14.596604 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134f163-6b60-458a-a53c-d1e6f2f36769 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:14.597174 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134f163-6b60-458a-a53c-d1e6f2f36769 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:14.597643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134f163-6b60-458a-a53c-d1e6f2f36769 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:14.599257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6134f163-6b60-458a-a53c-d1e6f2f36769 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:14.600889 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6134f163-6b60-458a-a53c-d1e6f2f36769 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:14.602910 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 148/299] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:14.608571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6134f163-6b60-458a-a53c-d1e6f2f36769 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:14.646702 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05752bd-720c-4e70-9e94-f46cf5034b59 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:14.676749 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:14.684787 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d32ce-71b1-47a0-b2c3-a0e3970f9b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:14.685740 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d32ce-71b1-47a0-b2c3-a0e3970f9b26 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:14.686029 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d32ce-71b1-47a0-b2c3-a0e3970f9b26 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:14.686309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d32ce-71b1-47a0-b2c3-a0e3970f9b26 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:14.696693 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d32ce-71b1-47a0-b2c3-a0e3970f9b26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:14.705025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-cd8d32ce-71b1-47a0-b2c3-a0e3970f9b26 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 13:32:14.705025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:14.705025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:14.705025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:14.705025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:14.705025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:14.705025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:14.705025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:14.705025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:14.705025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:14.705025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:14.705025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:14.705025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:14.705025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:14.705025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:14.705025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:14.706076 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:14.706076 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:14.706076 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:14.706076 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:14.706076 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:14.706076 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:14.706076 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:14.706076 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:14.706076 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:14.706076 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:14.706076 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:14.706076 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:14.706076 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:14.706076 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:14.706076 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:14.706076 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:14.706878 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:14.706878 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:14.706878 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:14.706878 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:14.706878 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:14.706878 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:14.706878 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:14.706878 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:14.706878 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:14.706878 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:14.706878 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 13:32:14.706878 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:14.706878 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 152/300] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:32:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:14.707743 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee81102-0223-487e-85ad-f480804b5612 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:14.708033 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee81102-0223-487e-85ad-f480804b5612 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:14.708311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee81102-0223-487e-85ad-f480804b5612 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:14.709754 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 153/301] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:14.713932 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee81102-0223-487e-85ad-f480804b5612 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:14.731234 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:14.745422 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6134f163-6b60-458a-a53c-d1e6f2f36769 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:14.792808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a36e7fc-a88e-4153-ae9d-2c5721fc3ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:14.793551 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36e7fc-a88e-4153-ae9d-2c5721fc3ea2 None admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:14.793842 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36e7fc-a88e-4153-ae9d-2c5721fc3ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:14.794122 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36e7fc-a88e-4153-ae9d-2c5721fc3ea2 None admin] PATH_INFO: `/v3/groups` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:14.794827 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a36e7fc-a88e-4153-ae9d-2c5721fc3ea2 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:14.796257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a36e7fc-a88e-4153-ae9d-2c5721fc3ea2 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:14.811400 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 149/302] 10.222.0.20 () {60 vars in 1134 bytes} [Wed Jul 6 13:32:14 2022] POST /identity/v3/groups => generated 229 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:32:14.812428 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c8547-8d24-4faa-88f8-87e005a81e14 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:14.812719 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c8547-8d24-4faa-88f8-87e005a81e14 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:14.813030 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c8547-8d24-4faa-88f8-87e005a81e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:14.838148 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-630c8547-8d24-4faa-88f8-87e005a81e14 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:14.839376 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-630c8547-8d24-4faa-88f8-87e005a81e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:14.858183 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee81102-0223-487e-85ad-f480804b5612 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:14.911996 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ee5bc-0deb-4726-9a6b-037b6adfb2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:14.913393 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ee5bc-0deb-4726-9a6b-037b6adfb2a0 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:14.914132 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ee5bc-0deb-4726-9a6b-037b6adfb2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:14.915069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ee5bc-0deb-4726-9a6b-037b6adfb2a0 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:14.916345 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ee5bc-0deb-4726-9a6b-037b6adfb2a0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:14.916896 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ee5bc-0deb-4726-9a6b-037b6adfb2a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:14.919597 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ee5bc-0deb-4726-9a6b-037b6adfb2a0 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:14.922983 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 150/303] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:14.935352 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 154/304] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jul 6 13:32:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:14.939423 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c8547-8d24-4faa-88f8-87e005a81e14 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:14.947128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ee5bc-0deb-4726-9a6b-037b6adfb2a0 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:15.009757 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:15.056315 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:15.084768 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ee5bc-0deb-4726-9a6b-037b6adfb2a0 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:15.084951 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c8547-8d24-4faa-88f8-87e005a81e14 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:15.124353 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e0531-a5a6-46d4-a36a-b1f6402a3734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:15.125208 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e0531-a5a6-46d4-a36a-b1f6402a3734 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:15.125492 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e0531-a5a6-46d4-a36a-b1f6402a3734 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:15.125767 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e0531-a5a6-46d4-a36a-b1f6402a3734 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:15.131349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59cafa-ac94-4835-bd86-1c5b20afb446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:15.132348 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59cafa-ac94-4835-bd86-1c5b20afb446 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:15.132631 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59cafa-ac94-4835-bd86-1c5b20afb446 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:15.132902 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59cafa-ac94-4835-bd86-1c5b20afb446 None admin] PATH_INFO: `/v3/users/23388bbd8e964d41a17d6f736d6199c2` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:15.137407 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e0531-a5a6-46d4-a36a-b1f6402a3734 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:15.146330 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-1e5e0531-a5a6-46d4-a36a-b1f6402a3734 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:32:15.146330 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:15.146330 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:15.146330 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:15.146330 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:15.146330 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:15.146330 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:15.146330 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:15.146330 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:15.146330 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:15.146330 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:15.146330 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:15.146330 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:15.146330 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:15.146330 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:15.146330 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:15.147215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:15.147215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:15.147215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:15.147215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:15.147215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:15.147215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:15.147215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:15.147215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:15.147215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:15.147215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:15.147215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:15.147215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:15.147215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:15.147215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:15.147215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:15.147215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:15.148074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:15.148074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:15.148074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:15.148074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:15.148074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:15.148074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:15.148074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:15.148074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:15.148074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:15.148074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:15.148074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:32:15.148074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:15.148687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab59cafa-ac94-4835-bd86-1c5b20afb446 None admin] RBAC: Authorizing `identity:get_user(user_id=23388bbd8e964d41a17d6f736d6199c2)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:15.148758 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 155/305] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:32:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:32:15.153331 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab59cafa-ac94-4835-bd86-1c5b20afb446 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:15.153331 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 151/306] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:32:14 2022] GET /identity/v3/users/23388bbd8e964d41a17d6f736d6199c2 => generated 293 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:15.158278 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e0531-a5a6-46d4-a36a-b1f6402a3734 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:15.158430 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59cafa-ac94-4835-bd86-1c5b20afb446 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:15.262147 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59cafa-ac94-4835-bd86-1c5b20afb446 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:15.284604 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e0531-a5a6-46d4-a36a-b1f6402a3734 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:15.306349 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:15.307898 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a39ece-87a1-40b4-a570-27cb6cd9c68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:15.308520 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a39ece-87a1-40b4-a570-27cb6cd9c68b None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:15.308780 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a39ece-87a1-40b4-a570-27cb6cd9c68b None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:15.309582 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a39ece-87a1-40b4-a570-27cb6cd9c68b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:15.310550 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a39ece-87a1-40b4-a570-27cb6cd9c68b None admin] RBAC: Adding query filter params (role.id=65ecdcc5a9d0487ba529e54c626e152f, user.id=23388bbd8e964d41a17d6f736d6199c2, scope.system=all) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:15.310893 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a39ece-87a1-40b4-a570-27cb6cd9c68b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:15.312352 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a39ece-87a1-40b4-a570-27cb6cd9c68b None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:15.318433 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 152/307] 10.222.0.20 () {56 vars in 1308 bytes} [Wed Jul 6 13:32:15 2022] GET /identity/v3/role_assignments?user.id=23388bbd8e964d41a17d6f736d6199c2&scope.system=all&role.id=65ecdcc5a9d0487ba529e54c626e152f => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:15.347285 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf37f74b-21f5-44df-be94-f33f2bfd3ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:15.347892 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37f74b-21f5-44df-be94-f33f2bfd3ddb None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:15.348176 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37f74b-21f5-44df-be94-f33f2bfd3ddb None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:15.348452 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37f74b-21f5-44df-be94-f33f2bfd3ddb None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:15.349252 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37f74b-21f5-44df-be94-f33f2bfd3ddb None admin] RBAC: Adding query filter params (name=member) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:15.349559 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37f74b-21f5-44df-be94-f33f2bfd3ddb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:15.350974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37f74b-21f5-44df-be94-f33f2bfd3ddb None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:15.362175 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 156/308] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jul 6 13:32:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:15.367813 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a39ece-87a1-40b4-a570-27cb6cd9c68b None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:15.372120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:15.483189 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a39ece-87a1-40b4-a570-27cb6cd9c68b None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:15.519743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e83e74-0812-4d33-884a-7ad6f55eb66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:15.520975 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e83e74-0812-4d33-884a-7ad6f55eb66f None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:15.521512 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e83e74-0812-4d33-884a-7ad6f55eb66f None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:15.521995 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e83e74-0812-4d33-884a-7ad6f55eb66f None admin] PATH_INFO: `/v3/users/7b713d8aedf846c7887f734bb55736b8` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:15.523425 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e83e74-0812-4d33-884a-7ad6f55eb66f None admin] RBAC: Authorizing `identity:get_user(user_id=7b713d8aedf846c7887f734bb55736b8)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:15.525115 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e83e74-0812-4d33-884a-7ad6f55eb66f None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:15.527266 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 153/309] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:32:15 2022] GET /identity/v3/users/7b713d8aedf846c7887f734bb55736b8 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:15.533608 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf37f74b-21f5-44df-be94-f33f2bfd3ddb None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:15.638289 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf37f74b-21f5-44df-be94-f33f2bfd3ddb None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:15.675004 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c1b648-f72d-4e8b-a58c-5f745950db1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:15.675907 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1b648-f72d-4e8b-a58c-5f745950db1d None admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:15.676187 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1b648-f72d-4e8b-a58c-5f745950db1d None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:15.676462 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1b648-f72d-4e8b-a58c-5f745950db1d None admin] PATH_INFO: `/v3/system/users/7b713d8aedf846c7887f734bb55736b8/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:15.678085 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c1b648-f72d-4e8b-a58c-5f745950db1d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7b713d8aedf846c7887f734bb55736b8, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:15.679453 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c1b648-f72d-4e8b-a58c-5f745950db1d None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:15.689966 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 157/310] 10.222.0.20 () {58 vars in 1334 bytes} [Wed Jul 6 13:32:15 2022] PUT /identity/v3/system/users/7b713d8aedf846c7887f734bb55736b8/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:32:15.858602 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6baf2ec-f395-4ed9-990a-eb7bd27fc186 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:15.858960 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6baf2ec-f395-4ed9-990a-eb7bd27fc186 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:15.859173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6baf2ec-f395-4ed9-990a-eb7bd27fc186 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:15.860683 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 154/311] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:15.866047 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c645e84-a868-4812-bc8d-65997f0dd948 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:15.866339 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c645e84-a868-4812-bc8d-65997f0dd948 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:15.866619 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c645e84-a868-4812-bc8d-65997f0dd948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:15.884422 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-2c645e84-a868-4812-bc8d-65997f0dd948 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:15.885634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-2c645e84-a868-4812-bc8d-65997f0dd948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:15.921116 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:15.963273 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 158/312] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:15.987794 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6baf2ec-f395-4ed9-990a-eb7bd27fc186 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:16.066232 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:16.099705 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6baf2ec-f395-4ed9-990a-eb7bd27fc186 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:16.135425 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a72ab-364a-41cb-8a8d-012624eabe11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:16.136233 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a72ab-364a-41cb-8a8d-012624eabe11 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:16.136516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a72ab-364a-41cb-8a8d-012624eabe11 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:16.136797 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a72ab-364a-41cb-8a8d-012624eabe11 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:16.148232 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17a72ab-364a-41cb-8a8d-012624eabe11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:16.156097 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-a17a72ab-364a-41cb-8a8d-012624eabe11 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 13:32:16.156097 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:16.156097 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:16.156097 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:16.156097 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:16.156097 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:16.156097 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:16.156097 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:16.156097 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:16.156097 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:16.156097 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:16.156097 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:16.156097 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:16.156097 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:16.156097 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:16.156097 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:16.156983 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:16.156983 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:16.156983 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:16.156983 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:16.156983 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:16.156983 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:16.156983 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:16.156983 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:16.156983 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:16.156983 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:16.156983 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:16.156983 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:16.156983 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:16.156983 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:16.156983 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:16.156983 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:16.157822 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:16.157822 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:16.157822 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:16.157822 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:16.157822 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:16.157822 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:16.157822 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:16.157822 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:16.157822 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:16.157822 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:16.157822 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 13:32:16.157822 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:16.157822 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 155/313] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:32:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:16.163792 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c645e84-a868-4812-bc8d-65997f0dd948 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:16.270256 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c645e84-a868-4812-bc8d-65997f0dd948 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:16.325722 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce26cb22-e00d-45c4-a66a-a78626173338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:16.327182 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26cb22-e00d-45c4-a66a-a78626173338 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:16.327688 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26cb22-e00d-45c4-a66a-a78626173338 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:16.328160 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26cb22-e00d-45c4-a66a-a78626173338 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:16.329160 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce26cb22-e00d-45c4-a66a-a78626173338 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:16.329666 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce26cb22-e00d-45c4-a66a-a78626173338 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:16.331242 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce26cb22-e00d-45c4-a66a-a78626173338 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:16.340393 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 159/314] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jul 6 13:32:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:16.348203 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a72ab-364a-41cb-8a8d-012624eabe11 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:16.450644 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a72ab-364a-41cb-8a8d-012624eabe11 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:16.480316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:16.486281 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59595cf5-5bc7-4778-97ff-d2ac94fc6c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:16.487173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59595cf5-5bc7-4778-97ff-d2ac94fc6c57 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:16.487459 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59595cf5-5bc7-4778-97ff-d2ac94fc6c57 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:16.487734 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59595cf5-5bc7-4778-97ff-d2ac94fc6c57 None admin] PATH_INFO: `/v3/users/122dbda8957f44d9a09145414cfb256e` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:16.488956 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59595cf5-5bc7-4778-97ff-d2ac94fc6c57 None admin] RBAC: Authorizing `identity:get_user(user_id=122dbda8957f44d9a09145414cfb256e)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:16.490513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59595cf5-5bc7-4778-97ff-d2ac94fc6c57 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:16.497575 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 156/315] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:32:16 2022] GET /identity/v3/users/122dbda8957f44d9a09145414cfb256e => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:16.498878 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce26cb22-e00d-45c4-a66a-a78626173338 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:16.532490 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:16.614371 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce26cb22-e00d-45c4-a66a-a78626173338 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:16.653904 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2824dc-653f-45e1-bc9c-3e7eca19d431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:16.654925 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2824dc-653f-45e1-bc9c-3e7eca19d431 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:16.655428 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2824dc-653f-45e1-bc9c-3e7eca19d431 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:16.655913 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2824dc-653f-45e1-bc9c-3e7eca19d431 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:16.724372 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2824dc-653f-45e1-bc9c-3e7eca19d431 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:16.726215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2824dc-653f-45e1-bc9c-3e7eca19d431 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:16.757364 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:16.827975 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 160/316] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:32:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:32:16.833499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59595cf5-5bc7-4778-97ff-d2ac94fc6c57 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:16.941764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59595cf5-5bc7-4778-97ff-d2ac94fc6c57 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:16.977702 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992e11a-bcf9-4196-8a5a-0395c3937a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:16.978575 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992e11a-bcf9-4196-8a5a-0395c3937a68 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:16.978857 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992e11a-bcf9-4196-8a5a-0395c3937a68 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:16.979134 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992e11a-bcf9-4196-8a5a-0395c3937a68 None admin] PATH_INFO: `/v3/projects/acd1ac1812eb4faeb54c342d511d42de` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:16.980083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924caea-d36a-48ec-bef4-db005ee5b4eb None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:16.980337 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4992e11a-bcf9-4196-8a5a-0395c3937a68 None admin] RBAC: Authorizing `identity:get_project(project_id=acd1ac1812eb4faeb54c342d511d42de)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:16.980765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924caea-d36a-48ec-bef4-db005ee5b4eb None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:16.981923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924caea-d36a-48ec-bef4-db005ee5b4eb None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:16.982570 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4992e11a-bcf9-4196-8a5a-0395c3937a68 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:16.984027 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 161/317] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:32:16.987912 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 157/318] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:32:16 2022] GET /identity/v3/projects/acd1ac1812eb4faeb54c342d511d42de => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:16.990237 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992e11a-bcf9-4196-8a5a-0395c3937a68 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:16.990455 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609bc64-d925-4e8f-a893-eeeb7304800f None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:16.990927 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609bc64-d925-4e8f-a893-eeeb7304800f None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:16.991393 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609bc64-d925-4e8f-a893-eeeb7304800f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:17.013841 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-5609bc64-d925-4e8f-a893-eeeb7304800f None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:17.015316 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-5609bc64-d925-4e8f-a893-eeeb7304800f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:17.078184 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:17.113027 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 162/319] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:17.125138 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992e11a-bcf9-4196-8a5a-0395c3937a68 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:17.129957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609bc64-d925-4e8f-a893-eeeb7304800f None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:17.166876 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4dfee8-bc5d-4cb7-a0e2-5059d2aa7a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:17.167536 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4dfee8-bc5d-4cb7-a0e2-5059d2aa7a52 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:17.167821 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4dfee8-bc5d-4cb7-a0e2-5059d2aa7a52 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:17.168090 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4dfee8-bc5d-4cb7-a0e2-5059d2aa7a52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:17.168961 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4dfee8-bc5d-4cb7-a0e2-5059d2aa7a52 None admin] RBAC: Adding query filter params (role.id=65ecdcc5a9d0487ba529e54c626e152f, scope.project.id=acd1ac1812eb4faeb54c342d511d42de, user.id=122dbda8957f44d9a09145414cfb256e) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:17.169310 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4dfee8-bc5d-4cb7-a0e2-5059d2aa7a52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:17.170923 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4dfee8-bc5d-4cb7-a0e2-5059d2aa7a52 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:17.178617 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 158/320] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:32:16 2022] GET /identity/v3/role_assignments?user.id=122dbda8957f44d9a09145414cfb256e&scope.project.id=acd1ac1812eb4faeb54c342d511d42de&role.id=65ecdcc5a9d0487ba529e54c626e152f => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:17.204130 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:17.254670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609bc64-d925-4e8f-a893-eeeb7304800f None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:17.305412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef836bc-a686-41d2-9734-bbd880236d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:17.306728 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef836bc-a686-41d2-9734-bbd880236d1a None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:17.307223 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef836bc-a686-41d2-9734-bbd880236d1a None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:17.307692 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef836bc-a686-41d2-9734-bbd880236d1a None admin] PATH_INFO: `/v3/domains/default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:17.309739 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef836bc-a686-41d2-9734-bbd880236d1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:17.311428 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef836bc-a686-41d2-9734-bbd880236d1a None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:17.313788 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 163/321] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:17.319723 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4dfee8-bc5d-4cb7-a0e2-5059d2aa7a52 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:17.428009 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4dfee8-bc5d-4cb7-a0e2-5059d2aa7a52 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:17.452363 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15db1f-807a-499d-ae5d-5aaadf40f5ff None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:17.452863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15db1f-807a-499d-ae5d-5aaadf40f5ff None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:17.452963 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15db1f-807a-499d-ae5d-5aaadf40f5ff None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:17.454556 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 164/322] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:32:17.459658 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59ca64-9e99-4426-a314-0eb58aea71a3 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:17.459948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59ca64-9e99-4426-a314-0eb58aea71a3 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:17.460228 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59ca64-9e99-4426-a314-0eb58aea71a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:17.461717 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:17.469167 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f62c94-1ae2-4cea-8186-4971a296d34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:17.469842 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f62c94-1ae2-4cea-8186-4971a296d34a None admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:17.470132 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f62c94-1ae2-4cea-8186-4971a296d34a None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:17.470412 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f62c94-1ae2-4cea-8186-4971a296d34a None admin] PATH_INFO: `/v3/groups` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:17.471127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f62c94-1ae2-4cea-8186-4971a296d34a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:17.472573 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f62c94-1ae2-4cea-8186-4971a296d34a None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:17.478120 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-da59ca64-9e99-4426-a314-0eb58aea71a3 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:17.481392 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-da59ca64-9e99-4426-a314-0eb58aea71a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:17.497038 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 159/323] 10.222.0.20 () {60 vars in 1134 bytes} [Wed Jul 6 13:32:17 2022] POST /identity/v3/groups => generated 226 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:32:17.567605 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 165/324] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 13:32:17.592051 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f62c94-1ae2-4cea-8186-4971a296d34a None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:17.703396 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f62c94-1ae2-4cea-8186-4971a296d34a None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:17.741495 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7addab3-7b8a-4181-b445-d922cf9ee39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:17.742633 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7addab3-7b8a-4181-b445-d922cf9ee39f None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:17.743142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7addab3-7b8a-4181-b445-d922cf9ee39f None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:17.743619 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7addab3-7b8a-4181-b445-d922cf9ee39f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:17.754766 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7addab3-7b8a-4181-b445-d922cf9ee39f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:17.763675 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-c7addab3-7b8a-4181-b445-d922cf9ee39f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 13:32:17.763675 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:17.763675 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:17.763675 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:17.763675 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:17.763675 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:17.763675 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:17.763675 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:17.763675 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:17.763675 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:17.763675 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:17.763675 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:17.763675 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:17.763675 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:17.763675 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:17.763675 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:17.764687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:17.764687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:17.764687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:17.764687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:17.764687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:17.764687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:17.764687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:17.764687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:17.764687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:17.764687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:17.764687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:17.764687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:17.764687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:17.764687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:17.764687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:17.764687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:17.765639 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:17.765639 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:17.765639 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:17.765639 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:17.765639 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:17.765639 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:17.765639 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:17.765639 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:17.765639 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:17.765639 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:17.765639 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 13:32:17.765639 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:17.767940 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 160/325] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:32:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:17.774182 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59ca64-9e99-4426-a314-0eb58aea71a3 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:17.834884 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:17.857702 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a42577-4809-40c1-8c4d-d166370f1a19 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:17.858315 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a42577-4809-40c1-8c4d-d166370f1a19 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:17.858792 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a42577-4809-40c1-8c4d-d166370f1a19 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:17.860458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 161/326] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:17.867403 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65991eae-3b51-4ff1-98da-18dd1a29f1ca None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:17.867836 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65991eae-3b51-4ff1-98da-18dd1a29f1ca None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:17.868193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65991eae-3b51-4ff1-98da-18dd1a29f1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:17.883436 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59ca64-9e99-4426-a314-0eb58aea71a3 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:17.910332 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-65991eae-3b51-4ff1-98da-18dd1a29f1ca None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:17.911547 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-65991eae-3b51-4ff1-98da-18dd1a29f1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:17.923275 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8edce-23e6-4b60-9f1b-7568029fe375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:17.924204 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8edce-23e6-4b60-9f1b-7568029fe375 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:17.924487 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8edce-23e6-4b60-9f1b-7568029fe375 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:17.924762 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8edce-23e6-4b60-9f1b-7568029fe375 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:17.925552 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8edce-23e6-4b60-9f1b-7568029fe375 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:17.926190 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8edce-23e6-4b60-9f1b-7568029fe375 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:17.927595 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8edce-23e6-4b60-9f1b-7568029fe375 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:17.938809 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 166/327] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jul 6 13:32:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:17.951546 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8edce-23e6-4b60-9f1b-7568029fe375 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:17.978161 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:18.017606 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 162/328] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:18.041999 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65991eae-3b51-4ff1-98da-18dd1a29f1ca None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:18.118991 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8edce-23e6-4b60-9f1b-7568029fe375 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:18.143124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:18.166873 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47baf3bf-ba01-45fb-a083-3c0b5a30ae8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:18.168286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47baf3bf-ba01-45fb-a083-3c0b5a30ae8c None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:18.168724 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47baf3bf-ba01-45fb-a083-3c0b5a30ae8c None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:18.169153 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47baf3bf-ba01-45fb-a083-3c0b5a30ae8c None admin] PATH_INFO: `/v3/users/23388bbd8e964d41a17d6f736d6199c2` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:18.173744 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47baf3bf-ba01-45fb-a083-3c0b5a30ae8c None admin] RBAC: Authorizing `identity:get_user(user_id=23388bbd8e964d41a17d6f736d6199c2)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:18.175850 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47baf3bf-ba01-45fb-a083-3c0b5a30ae8c None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:18.179590 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 167/329] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:32:17 2022] GET /identity/v3/users/23388bbd8e964d41a17d6f736d6199c2 => generated 293 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:18.186263 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47baf3bf-ba01-45fb-a083-3c0b5a30ae8c None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:18.206392 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65991eae-3b51-4ff1-98da-18dd1a29f1ca None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:18.247213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfebedd5-8603-4cd7-9101-a801241d086e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:18.248285 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfebedd5-8603-4cd7-9101-a801241d086e None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:18.248706 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfebedd5-8603-4cd7-9101-a801241d086e None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:18.249143 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfebedd5-8603-4cd7-9101-a801241d086e None admin] PATH_INFO: `/v3/roles/member` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:18.260130 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfebedd5-8603-4cd7-9101-a801241d086e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:18.268271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-cfebedd5-8603-4cd7-9101-a801241d086e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:32:18.268271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:18.268271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:18.268271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:18.268271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:18.268271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:18.268271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:18.268271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:18.268271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:18.268271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:18.268271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:18.268271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:18.268271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:18.268271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:18.268271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:18.268271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:18.269400 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:18.269400 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:18.269400 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:18.269400 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:18.269400 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:18.269400 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:18.269400 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:18.269400 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:18.269400 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:18.269400 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:18.269400 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:18.269400 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:18.269400 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:18.269400 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:18.269400 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:18.269400 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:18.270405 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:18.270405 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:18.270405 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:18.270405 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:18.270405 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:18.270405 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:18.270405 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:18.270405 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:18.270405 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:18.270405 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:18.270405 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:32:18.270405 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:18.272787 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 163/330] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:32:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:32:18.279354 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfebedd5-8603-4cd7-9101-a801241d086e None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:18.329343 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47baf3bf-ba01-45fb-a083-3c0b5a30ae8c None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:18.344780 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:18.407233 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfebedd5-8603-4cd7-9101-a801241d086e None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:18.418160 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03a5a1-a55a-4a30-8326-21ad1e0d48d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:18.419931 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03a5a1-a55a-4a30-8326-21ad1e0d48d2 None admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:18.420533 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03a5a1-a55a-4a30-8326-21ad1e0d48d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:18.421174 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03a5a1-a55a-4a30-8326-21ad1e0d48d2 None admin] PATH_INFO: `/v3/system/users/23388bbd8e964d41a17d6f736d6199c2/roles/65ecdcc5a9d0487ba529e54c626e152f` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:18.426279 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03a5a1-a55a-4a30-8326-21ad1e0d48d2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=23388bbd8e964d41a17d6f736d6199c2, role_id=65ecdcc5a9d0487ba529e54c626e152f)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:18.427923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03a5a1-a55a-4a30-8326-21ad1e0d48d2 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:18.447548 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86291591-053a-4752-8c5d-069e71f4717f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:18.448488 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86291591-053a-4752-8c5d-069e71f4717f None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:18.448935 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86291591-053a-4752-8c5d-069e71f4717f None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:18.449366 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86291591-053a-4752-8c5d-069e71f4717f None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:18.451157 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86291591-053a-4752-8c5d-069e71f4717f None admin] RBAC: Adding query filter params (name=member) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:18.451777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86291591-053a-4752-8c5d-069e71f4717f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:18.452661 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 168/331] 10.222.0.20 () {58 vars in 1334 bytes} [Wed Jul 6 13:32:18 2022] PUT /identity/v3/system/users/23388bbd8e964d41a17d6f736d6199c2/roles/65ecdcc5a9d0487ba529e54c626e152f => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:32:18.454375 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86291591-053a-4752-8c5d-069e71f4717f None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:18.467298 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 164/332] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jul 6 13:32:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:18.471649 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03a5a1-a55a-4a30-8326-21ad1e0d48d2 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:18.531051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:18.589494 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03a5a1-a55a-4a30-8326-21ad1e0d48d2 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:18.627516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01a74f-32c7-43ea-acd6-f77f04393bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:18.628744 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01a74f-32c7-43ea-acd6-f77f04393bc3 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:18.629313 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01a74f-32c7-43ea-acd6-f77f04393bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:18.629726 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01a74f-32c7-43ea-acd6-f77f04393bc3 None admin] PATH_INFO: `/v3/users/7b713d8aedf846c7887f734bb55736b8` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:18.631118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed01a74f-32c7-43ea-acd6-f77f04393bc3 None admin] RBAC: Authorizing `identity:get_user(user_id=7b713d8aedf846c7887f734bb55736b8)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:18.632700 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed01a74f-32c7-43ea-acd6-f77f04393bc3 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:18.639915 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 169/333] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:32:18 2022] GET /identity/v3/users/7b713d8aedf846c7887f734bb55736b8 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:18.641457 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86291591-053a-4752-8c5d-069e71f4717f None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:18.777266 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86291591-053a-4752-8c5d-069e71f4717f None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:18.779902 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:18.817211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59732c-40a2-4172-a278-0521a1dc31ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:18.817796 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59732c-40a2-4172-a278-0521a1dc31ef None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:18.818081 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59732c-40a2-4172-a278-0521a1dc31ef None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:18.818355 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59732c-40a2-4172-a278-0521a1dc31ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:18.819195 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd59732c-40a2-4172-a278-0521a1dc31ef None admin] RBAC: Adding query filter params (role.id=b3244d1d9f004c1b8d3594fd30da1024, user.id=7b713d8aedf846c7887f734bb55736b8, scope.system=all) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:18.819509 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd59732c-40a2-4172-a278-0521a1dc31ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:18.821082 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd59732c-40a2-4172-a278-0521a1dc31ef None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:18.834852 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 165/334] 10.222.0.20 () {56 vars in 1308 bytes} [Wed Jul 6 13:32:18 2022] GET /identity/v3/role_assignments?user.id=7b713d8aedf846c7887f734bb55736b8&scope.system=all&role.id=b3244d1d9f004c1b8d3594fd30da1024 => generated 510 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:20.042523 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ed585-d828-40d9-863a-d092252e6d76 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:20.043025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ed585-d828-40d9-863a-d092252e6d76 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:20.043182 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ed585-d828-40d9-863a-d092252e6d76 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:20.045263 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 170/335] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:20.051912 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eebe15-b400-4a11-a005-3f3e5b0bf90d None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:20.052355 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eebe15-b400-4a11-a005-3f3e5b0bf90d None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:20.052537 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eebe15-b400-4a11-a005-3f3e5b0bf90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:20.070715 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-d5eebe15-b400-4a11-a005-3f3e5b0bf90d None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:20.071912 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-d5eebe15-b400-4a11-a005-3f3e5b0bf90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:20.146934 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 166/336] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 13:32:20.172423 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ed585-d828-40d9-863a-d092252e6d76 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:20.287199 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ed585-d828-40d9-863a-d092252e6d76 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:20.310389 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:20.331949 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d822f3-d0c1-4a51-b244-f19043dbac93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:20.332769 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d822f3-d0c1-4a51-b244-f19043dbac93 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:20.333091 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d822f3-d0c1-4a51-b244-f19043dbac93 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:20.333349 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d822f3-d0c1-4a51-b244-f19043dbac93 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:20.353892 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d822f3-d0c1-4a51-b244-f19043dbac93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:20.362268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-e1d822f3-d0c1-4a51-b244-f19043dbac93 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 13:32:20.362268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:20.362268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:20.362268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:20.362268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:20.362268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:20.362268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:20.362268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:20.362268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:20.362268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:20.362268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:20.362268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:20.362268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:20.362268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:20.362268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:20.362268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:20.363135 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:20.363135 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:20.363135 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:20.363135 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:20.363135 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:20.363135 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:20.363135 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:20.363135 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:20.363135 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:20.363135 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:20.363135 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:20.363135 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:20.363135 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:20.363135 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:20.363135 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:20.363135 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:20.363936 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:20.363936 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:20.363936 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:20.363936 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:20.363936 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:20.363936 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:20.363936 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:20.363936 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:20.363936 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:20.363936 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:20.363936 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 13:32:20.363936 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:20.363936 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 171/337] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jul 6 13:32:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:20.369236 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eebe15-b400-4a11-a005-3f3e5b0bf90d None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:20.476317 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:20.486490 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eebe15-b400-4a11-a005-3f3e5b0bf90d None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:20.526920 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351c097-c2c1-4e2b-8172-c7f3ce3a7ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:20.528114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351c097-c2c1-4e2b-8172-c7f3ce3a7ad4 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:20.528423 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351c097-c2c1-4e2b-8172-c7f3ce3a7ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:20.528905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351c097-c2c1-4e2b-8172-c7f3ce3a7ad4 None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:20.529590 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351c097-c2c1-4e2b-8172-c7f3ce3a7ad4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:20.529904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351c097-c2c1-4e2b-8172-c7f3ce3a7ad4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:20.531296 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351c097-c2c1-4e2b-8172-c7f3ce3a7ad4 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:20.537981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c840832-c29b-4f20-86ff-65cc81bf654b None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:20.538268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c840832-c29b-4f20-86ff-65cc81bf654b None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:20.538550 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c840832-c29b-4f20-86ff-65cc81bf654b None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:20.539926 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 172/338] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:32:20.545172 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a77a12-8202-4815-bed7-20f26f591f21 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:20.545408 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a77a12-8202-4815-bed7-20f26f591f21 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:20.545674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a77a12-8202-4815-bed7-20f26f591f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:20.552122 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 167/339] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:32:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:20.560362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351c097-c2c1-4e2b-8172-c7f3ce3a7ad4 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:20.574226 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-e2a77a12-8202-4815-bed7-20f26f591f21 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:20.575879 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-e2a77a12-8202-4815-bed7-20f26f591f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:20.732252 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351c097-c2c1-4e2b-8172-c7f3ce3a7ad4 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:20.742385 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 173/340] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:20.778258 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a77a12-8202-4815-bed7-20f26f591f21 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:20.811377 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd865743-6b78-4f9d-8aa8-6c451e35fb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:20.811982 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd865743-6b78-4f9d-8aa8-6c451e35fb21 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:20.812271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd865743-6b78-4f9d-8aa8-6c451e35fb21 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:20.812555 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd865743-6b78-4f9d-8aa8-6c451e35fb21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:20.856413 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:20.962101 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd865743-6b78-4f9d-8aa8-6c451e35fb21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:20.963675 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd865743-6b78-4f9d-8aa8-6c451e35fb21 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:21.020844 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a77a12-8202-4815-bed7-20f26f591f21 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:21.040274 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:21.097650 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50d849-61d6-4b13-b73e-daf7c13c6b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:21.098404 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50d849-61d6-4b13-b73e-daf7c13c6b72 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:21.098685 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50d849-61d6-4b13-b73e-daf7c13c6b72 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:21.098945 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50d849-61d6-4b13-b73e-daf7c13c6b72 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:21.129994 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e50d849-61d6-4b13-b73e-daf7c13c6b72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:21.138187 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-9e50d849-61d6-4b13-b73e-daf7c13c6b72 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:32:21.138187 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:21.138187 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:21.138187 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:21.138187 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:21.138187 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:21.138187 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:21.138187 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:21.138187 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:21.138187 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:21.138187 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:21.138187 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:21.138187 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:21.138187 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:21.138187 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:21.138187 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:21.139100 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:21.139100 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:21.139100 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:21.139100 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:21.139100 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:21.139100 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:21.139100 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:21.139100 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:21.139100 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:21.139100 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:21.139100 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:21.139100 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:21.139100 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:21.139100 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:21.139100 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:21.139100 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:21.139916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:21.139916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:21.139916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:21.139916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:21.139916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:21.139916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:21.139916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:21.139916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:21.139916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:21.139916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:21.139916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:32:21.139916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:21.144300 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 174/341] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:32:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:21.145196 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50d849-61d6-4b13-b73e-daf7c13c6b72 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:21.200298 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 168/342] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:32:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 640 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:32:21.207590 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd865743-6b78-4f9d-8aa8-6c451e35fb21 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:21.287694 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:21.368337 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd865743-6b78-4f9d-8aa8-6c451e35fb21 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:21.376282 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50d849-61d6-4b13-b73e-daf7c13c6b72 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:21.422504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ddcd2-344c-4657-8b5d-bad9a531db4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:21.423519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ddcd2-344c-4657-8b5d-bad9a531db4e None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:21.423804 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ddcd2-344c-4657-8b5d-bad9a531db4e None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:21.424336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ddcd2-344c-4657-8b5d-bad9a531db4e None admin] PATH_INFO: `/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:21.426375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96616302-bba4-4964-85ef-f50c0b50a207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:21.427063 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96616302-bba4-4964-85ef-f50c0b50a207 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:21.427347 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96616302-bba4-4964-85ef-f50c0b50a207 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:21.427619 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96616302-bba4-4964-85ef-f50c0b50a207 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:21.428392 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96616302-bba4-4964-85ef-f50c0b50a207 None admin] RBAC: Adding query filter params (name=member) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:21.428699 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96616302-bba4-4964-85ef-f50c0b50a207 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:21.429892 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831ddcd2-344c-4657-8b5d-bad9a531db4e None admin] RBAC: Authorizing `identity:get_project(project_id=202d7ba5c7f34872aa3ab82e5d2b2f93)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:21.430131 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96616302-bba4-4964-85ef-f50c0b50a207 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:21.431366 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831ddcd2-344c-4657-8b5d-bad9a531db4e None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:21.433533 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 169/343] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:32:21 2022] GET /identity/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:21.439824 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ddcd2-344c-4657-8b5d-bad9a531db4e None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:21.445295 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 175/344] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jul 6 13:32:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:21.452360 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96616302-bba4-4964-85ef-f50c0b50a207 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:21.497448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:21.576951 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96616302-bba4-4964-85ef-f50c0b50a207 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:21.585805 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ddcd2-344c-4657-8b5d-bad9a531db4e None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:21.613517 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d67ea-4d1d-4742-b846-84672e8f9f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:21.614171 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d67ea-4d1d-4742-b846-84672e8f9f59 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:21.614453 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d67ea-4d1d-4742-b846-84672e8f9f59 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:21.614726 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d67ea-4d1d-4742-b846-84672e8f9f59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:21.618321 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:21.645939 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ffa90-0195-4e73-ae25-baa7bb1a8ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:21.646801 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ffa90-0195-4e73-ae25-baa7bb1a8ca6 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:21.647093 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ffa90-0195-4e73-ae25-baa7bb1a8ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:21.647380 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ffa90-0195-4e73-ae25-baa7bb1a8ca6 None admin] PATH_INFO: `/v3/groups/eea5676f62f245f0b9d8d1594c15006c` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:21.657727 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ffa90-0195-4e73-ae25-baa7bb1a8ca6 None admin] RBAC: Authorizing `identity:get_group(group_id=eea5676f62f245f0b9d8d1594c15006c)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:21.659597 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ffa90-0195-4e73-ae25-baa7bb1a8ca6 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:21.661955 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 170/345] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Jul 6 13:32:21 2022] GET /identity/v3/groups/eea5676f62f245f0b9d8d1594c15006c => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:21.663429 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40636461-8b60-4d04-91ac-6a572e589cd0 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:21.663935 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40636461-8b60-4d04-91ac-6a572e589cd0 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:21.664420 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40636461-8b60-4d04-91ac-6a572e589cd0 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:21.670081 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 171/346] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:21.674505 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40636461-8b60-4d04-91ac-6a572e589cd0 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:21.689394 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d67ea-4d1d-4742-b846-84672e8f9f59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:21.690850 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d67ea-4d1d-4742-b846-84672e8f9f59 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:21.724972 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:21.815850 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40636461-8b60-4d04-91ac-6a572e589cd0 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:21.827594 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 176/347] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:32:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 06 13:32:21.834363 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b561d1-d750-4e1c-bd93-5b889e989b14 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:21.834662 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b561d1-d750-4e1c-bd93-5b889e989b14 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:21.834941 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b561d1-d750-4e1c-bd93-5b889e989b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:21.870478 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30148701-bc03-4054-860d-4aa27dd4e725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:21.870704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-53b561d1-d750-4e1c-bd93-5b889e989b14 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:21.872155 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30148701-bc03-4054-860d-4aa27dd4e725 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:21.872373 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-53b561d1-d750-4e1c-bd93-5b889e989b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:21.872907 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30148701-bc03-4054-860d-4aa27dd4e725 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:21.873415 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30148701-bc03-4054-860d-4aa27dd4e725 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:21.874764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30148701-bc03-4054-860d-4aa27dd4e725 None admin] RBAC: Adding query filter params (group.id=eea5676f62f245f0b9d8d1594c15006c, role.id=534400e7b28546a8a04c08f825c28566, scope.project.id=202d7ba5c7f34872aa3ab82e5d2b2f93) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:21.875336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30148701-bc03-4054-860d-4aa27dd4e725 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:21.877068 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30148701-bc03-4054-860d-4aa27dd4e725 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:21.883469 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 172/348] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Jul 6 13:32:21 2022] GET /identity/v3/role_assignments?group.id=eea5676f62f245f0b9d8d1594c15006c&scope.project.id=202d7ba5c7f34872aa3ab82e5d2b2f93&role.id=534400e7b28546a8a04c08f825c28566 => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:21.890580 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30148701-bc03-4054-860d-4aa27dd4e725 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:21.957523 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:21.983577 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 177/349] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:21.985675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7901ad-5503-4406-b3cd-8677e9815188 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:21.985675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7901ad-5503-4406-b3cd-8677e9815188 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:21.985675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7901ad-5503-4406-b3cd-8677e9815188 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:21.986916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 178/350] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:21.993606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313bdca-4995-49aa-9695-3f0e81554b52 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:21.993606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313bdca-4995-49aa-9695-3f0e81554b52 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:21.993941 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313bdca-4995-49aa-9695-3f0e81554b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:22.019080 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-f313bdca-4995-49aa-9695-3f0e81554b52 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:22.020248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-f313bdca-4995-49aa-9695-3f0e81554b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:22.021492 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30148701-bc03-4054-860d-4aa27dd4e725 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:22.061679 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeccaf2-aa1c-46f5-993d-a39b4b84abe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:22.063030 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeccaf2-aa1c-46f5-993d-a39b4b84abe2 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:22.063512 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeccaf2-aa1c-46f5-993d-a39b4b84abe2 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:22.063993 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeccaf2-aa1c-46f5-993d-a39b4b84abe2 None admin] PATH_INFO: `/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:22.065674 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeccaf2-aa1c-46f5-993d-a39b4b84abe2 None admin] RBAC: Authorizing `identity:get_project(project_id=202d7ba5c7f34872aa3ab82e5d2b2f93)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:22.067441 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeccaf2-aa1c-46f5-993d-a39b4b84abe2 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:22.069940 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 173/351] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:32:21 2022] GET /identity/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:22.070989 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeccaf2-aa1c-46f5-993d-a39b4b84abe2 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:22.112800 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:22.137343 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 179/352] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:22.138242 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313bdca-4995-49aa-9695-3f0e81554b52 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:22.196891 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeccaf2-aa1c-46f5-993d-a39b4b84abe2 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:22.218346 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:22.239363 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca81d72-2c9f-4460-be1c-e6278b84fe61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:22.240582 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca81d72-2c9f-4460-be1c-e6278b84fe61 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:22.241171 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca81d72-2c9f-4460-be1c-e6278b84fe61 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:22.241654 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca81d72-2c9f-4460-be1c-e6278b84fe61 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:22.254723 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca81d72-2c9f-4460-be1c-e6278b84fe61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:22.265061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-4ca81d72-2c9f-4460-be1c-e6278b84fe61 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 13:32:22.265061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:22.265061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:22.265061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:22.265061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:22.265061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:22.265061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:22.265061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:22.265061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:22.265061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:22.265061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:22.265061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:22.265061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:22.265061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:22.265061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:22.265061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:22.266046 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:22.266046 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:22.266046 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:22.266046 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:22.266046 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:22.266046 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:22.266046 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:22.266046 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:22.266046 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:22.266046 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:22.266046 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:22.266046 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:22.266046 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:22.266046 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:22.266046 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:22.266046 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:22.266885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:22.266885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:22.266885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:22.266885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:22.266885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:22.266885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:22.266885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:22.266885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:22.266885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:22.266885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:22.266885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 13:32:22.266885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:22.268379 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 174/353] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jul 6 13:32:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:22.272965 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0652a3-d594-481c-a7bd-744caaf289f4 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:22.272965 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0652a3-d594-481c-a7bd-744caaf289f4 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:22.272965 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0652a3-d594-481c-a7bd-744caaf289f4 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:22.272965 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 175/354] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:22.272965 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0652a3-d594-481c-a7bd-744caaf289f4 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:22.276597 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313bdca-4995-49aa-9695-3f0e81554b52 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:22.325148 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a748d236-a2eb-4e23-b2e5-8e4d18a52ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:22.326498 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748d236-a2eb-4e23-b2e5-8e4d18a52ec9 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:22.327041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748d236-a2eb-4e23-b2e5-8e4d18a52ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:22.327534 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748d236-a2eb-4e23-b2e5-8e4d18a52ec9 None admin] PATH_INFO: `/v3/groups/eea5676f62f245f0b9d8d1594c15006c` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:22.329062 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a748d236-a2eb-4e23-b2e5-8e4d18a52ec9 None admin] RBAC: Authorizing `identity:get_group(group_id=eea5676f62f245f0b9d8d1594c15006c)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:22.330801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a748d236-a2eb-4e23-b2e5-8e4d18a52ec9 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:22.332694 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 180/355] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Jul 6 13:32:22 2022] GET /identity/v3/groups/eea5676f62f245f0b9d8d1594c15006c => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:22.333761 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a748d236-a2eb-4e23-b2e5-8e4d18a52ec9 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:22.406973 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:22.409372 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0652a3-d594-481c-a7bd-744caaf289f4 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:22.450690 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba58e87-ce54-40d9-8a62-4a056b8a6cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:22.451887 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba58e87-ce54-40d9-8a62-4a056b8a6cb9 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:22.452496 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba58e87-ce54-40d9-8a62-4a056b8a6cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:22.453123 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba58e87-ce54-40d9-8a62-4a056b8a6cb9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:22.456703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a748d236-a2eb-4e23-b2e5-8e4d18a52ec9 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:22.465500 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba58e87-ce54-40d9-8a62-4a056b8a6cb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:22.474213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-5ba58e87-ce54-40d9-8a62-4a056b8a6cb9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:32:22.474213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:22.474213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:22.474213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:22.474213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:22.474213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:22.474213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:22.474213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:22.474213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:22.474213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:22.474213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:22.474213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:22.474213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:22.474213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:22.474213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:22.474213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:22.475102 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:22.475102 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:22.475102 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:22.475102 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:22.475102 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:22.475102 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:22.475102 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:22.475102 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:22.475102 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:22.475102 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:22.475102 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:22.475102 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:22.475102 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:22.475102 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:22.475102 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:22.475102 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:22.475905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:22.475905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:22.475905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:22.475905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:22.475905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:22.475905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:22.475905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:22.475905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:22.475905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:22.475905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:22.475905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:32:22.475905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:22.478550 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 176/356] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:32:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:22.479967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76407c1-0ffd-4252-b404-cb7fae54e5a1 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:22.480452 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76407c1-0ffd-4252-b404-cb7fae54e5a1 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:22.481470 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76407c1-0ffd-4252-b404-cb7fae54e5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:22.502010 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-a76407c1-0ffd-4252-b404-cb7fae54e5a1 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:22.503468 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b6322-48de-4110-ae96-0754af6361e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:22.503830 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a76407c1-0ffd-4252-b404-cb7fae54e5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:22.504156 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b6322-48de-4110-ae96-0754af6361e4 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:22.504440 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b6322-48de-4110-ae96-0754af6361e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:22.504721 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b6322-48de-4110-ae96-0754af6361e4 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:22.505569 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b6322-48de-4110-ae96-0754af6361e4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:22.505919 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b6322-48de-4110-ae96-0754af6361e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:22.507289 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b6322-48de-4110-ae96-0754af6361e4 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:22.516598 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 181/357] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:32:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:22.520414 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b6322-48de-4110-ae96-0754af6361e4 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:22.544947 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:22.590814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 177/358] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:22.592200 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76407c1-0ffd-4252-b404-cb7fae54e5a1 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:22.650410 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b6322-48de-4110-ae96-0754af6361e4 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:22.676230 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:22.698541 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599cc72-69d7-4c78-a677-2b0f9c7175c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:22.699204 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599cc72-69d7-4c78-a677-2b0f9c7175c1 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:22.699529 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599cc72-69d7-4c78-a677-2b0f9c7175c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:22.699806 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599cc72-69d7-4c78-a677-2b0f9c7175c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:22.700852 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0599cc72-69d7-4c78-a677-2b0f9c7175c1 None admin] RBAC: Adding query filter params (group.id=eea5676f62f245f0b9d8d1594c15006c, role.id=b3244d1d9f004c1b8d3594fd30da1024, scope.project.id=202d7ba5c7f34872aa3ab82e5d2b2f93) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:22.701223 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0599cc72-69d7-4c78-a677-2b0f9c7175c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:22.702653 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0599cc72-69d7-4c78-a677-2b0f9c7175c1 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:22.708985 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 182/359] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Jul 6 13:32:22 2022] GET /identity/v3/role_assignments?group.id=eea5676f62f245f0b9d8d1594c15006c&scope.project.id=202d7ba5c7f34872aa3ab82e5d2b2f93&role.id=b3244d1d9f004c1b8d3594fd30da1024 => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:22.709913 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599cc72-69d7-4c78-a677-2b0f9c7175c1 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:22.712128 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76407c1-0ffd-4252-b404-cb7fae54e5a1 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:22.757966 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c9b98f-d2cd-4467-b366-d4eff111b580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:22.758675 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9b98f-d2cd-4467-b366-d4eff111b580 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:22.758961 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9b98f-d2cd-4467-b366-d4eff111b580 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:22.759286 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9b98f-d2cd-4467-b366-d4eff111b580 None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:22.760117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c9b98f-d2cd-4467-b366-d4eff111b580 None admin] RBAC: Adding query filter params (name=member) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:22.761096 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c9b98f-d2cd-4467-b366-d4eff111b580 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:22.762625 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c9b98f-d2cd-4467-b366-d4eff111b580 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:22.778896 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 178/360] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jul 6 13:32:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:22.779789 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c9b98f-d2cd-4467-b366-d4eff111b580 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:22.836681 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:22.873178 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599cc72-69d7-4c78-a677-2b0f9c7175c1 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:22.915789 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8fdc8-1b86-43d4-a57e-25ad12012d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:22.916453 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8fdc8-1b86-43d4-a57e-25ad12012d0b None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:22.916735 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8fdc8-1b86-43d4-a57e-25ad12012d0b None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:22.917027 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8fdc8-1b86-43d4-a57e-25ad12012d0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:22.927107 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c9b98f-d2cd-4467-b366-d4eff111b580 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:22.970448 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44435c11-d34e-471e-b993-fc59aad88c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:22.971271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44435c11-d34e-471e-b993-fc59aad88c71 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:22.971556 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44435c11-d34e-471e-b993-fc59aad88c71 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:22.971833 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44435c11-d34e-471e-b993-fc59aad88c71 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:22.984673 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:22.986261 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44435c11-d34e-471e-b993-fc59aad88c71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:22.995513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-44435c11-d34e-471e-b993-fc59aad88c71 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 13:32:22.995513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:22.995513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:22.995513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:22.995513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:22.995513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:22.995513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:22.995513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:22.995513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:22.995513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:22.995513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:22.995513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:22.995513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:22.995513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:22.995513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:22.995513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:22.997120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:22.997120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:22.997120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:22.997120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:22.997120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:22.997120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:22.997120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:22.997120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:22.997120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:22.997120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:22.997120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:22.997120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:22.997120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:22.997120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:22.997120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:22.997120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:22.997972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:22.997972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:22.997972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:22.997972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:22.997972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:22.997972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:22.997972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:22.997972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:22.997972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:22.997972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:22.997972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 13:32:22.997972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:22.998859 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 179/361] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:32:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:22.999357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee09be-e719-4839-af1f-67a1dd437053 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:22.999647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee09be-e719-4839-af1f-67a1dd437053 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:22.999925 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee09be-e719-4839-af1f-67a1dd437053 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:23.001228 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 180/362] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:23.002041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee09be-e719-4839-af1f-67a1dd437053 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:23.006585 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c8fdc8-1b86-43d4-a57e-25ad12012d0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:23.008414 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c8fdc8-1b86-43d4-a57e-25ad12012d0b None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:23.114228 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:23.120684 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 183/363] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:32:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:32:23.122409 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8fdc8-1b86-43d4-a57e-25ad12012d0b None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:23.127166 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee09be-e719-4839-af1f-67a1dd437053 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:23.171908 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ad181-97c6-46f2-8ae1-87968c97dc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:23.173306 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ad181-97c6-46f2-8ae1-87968c97dc75 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:23.173595 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ad181-97c6-46f2-8ae1-87968c97dc75 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:23.174118 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ad181-97c6-46f2-8ae1-87968c97dc75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:23.226722 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:23.240819 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8fdc8-1b86-43d4-a57e-25ad12012d0b None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:23.269751 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45ad181-97c6-46f2-8ae1-87968c97dc75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:23.271157 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45ad181-97c6-46f2-8ae1-87968c97dc75 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:23.281640 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3d7f4-59fb-49d3-aa3c-e62e2f1329d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:23.282356 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3d7f4-59fb-49d3-aa3c-e62e2f1329d2 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:23.282646 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3d7f4-59fb-49d3-aa3c-e62e2f1329d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:23.282926 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3d7f4-59fb-49d3-aa3c-e62e2f1329d2 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:23.283798 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3d7f4-59fb-49d3-aa3c-e62e2f1329d2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:23.284107 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3d7f4-59fb-49d3-aa3c-e62e2f1329d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:23.285636 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3d7f4-59fb-49d3-aa3c-e62e2f1329d2 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:23.298853 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 184/364] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jul 6 13:32:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:23.300409 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930c362-e696-43a9-90c9-cad85ac30757 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:23.300697 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930c362-e696-43a9-90c9-cad85ac30757 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:23.300975 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930c362-e696-43a9-90c9-cad85ac30757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:23.329741 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-1930c362-e696-43a9-90c9-cad85ac30757 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:23.331438 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-1930c362-e696-43a9-90c9-cad85ac30757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:23.395140 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:23.407538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 181/365] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:32:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:32:23.408380 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ad181-97c6-46f2-8ae1-87968c97dc75 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:23.421202 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 185/366] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:23.422074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1930c362-e696-43a9-90c9-cad85ac30757 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:23.524175 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ad181-97c6-46f2-8ae1-87968c97dc75 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:23.529492 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:23.555770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1930c362-e696-43a9-90c9-cad85ac30757 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:23.562873 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda3c10-ca26-4782-8f37-98b7aae51150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:23.563745 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda3c10-ca26-4782-8f37-98b7aae51150 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:23.564027 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda3c10-ca26-4782-8f37-98b7aae51150 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:23.564303 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda3c10-ca26-4782-8f37-98b7aae51150 None admin] PATH_INFO: `/v3/projects/acd1ac1812eb4faeb54c342d511d42de` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:23.565831 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda3c10-ca26-4782-8f37-98b7aae51150 None admin] RBAC: Authorizing `identity:get_project(project_id=acd1ac1812eb4faeb54c342d511d42de)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:23.567338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda3c10-ca26-4782-8f37-98b7aae51150 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:23.569361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 182/367] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:32:23 2022] GET /identity/v3/projects/acd1ac1812eb4faeb54c342d511d42de => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:23.570144 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda3c10-ca26-4782-8f37-98b7aae51150 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:23.593939 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97889cdc-f666-474d-82c4-ec7b4fa4f12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:23.594819 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97889cdc-f666-474d-82c4-ec7b4fa4f12b None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:23.595101 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97889cdc-f666-474d-82c4-ec7b4fa4f12b None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:23.595433 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97889cdc-f666-474d-82c4-ec7b4fa4f12b None admin] PATH_INFO: `/v3/users/23388bbd8e964d41a17d6f736d6199c2` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:23.596710 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97889cdc-f666-474d-82c4-ec7b4fa4f12b None admin] RBAC: Authorizing `identity:get_user(user_id=23388bbd8e964d41a17d6f736d6199c2)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:23.598191 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97889cdc-f666-474d-82c4-ec7b4fa4f12b None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:23.599899 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 186/368] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:32:23 2022] GET /identity/v3/users/23388bbd8e964d41a17d6f736d6199c2 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:23.600714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97889cdc-f666-474d-82c4-ec7b4fa4f12b None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:23.642854 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:23.691164 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda3c10-ca26-4782-8f37-98b7aae51150 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:23.711595 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97889cdc-f666-474d-82c4-ec7b4fa4f12b None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:23.726577 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa86053-2b1d-4ba4-83ae-957fdc8fd152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:23.727441 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa86053-2b1d-4ba4-83ae-957fdc8fd152 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:23.727724 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa86053-2b1d-4ba4-83ae-957fdc8fd152 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:23.728003 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa86053-2b1d-4ba4-83ae-957fdc8fd152 None admin] PATH_INFO: `/v3/projects/acd1ac1812eb4faeb54c342d511d42de` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:23.729218 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa86053-2b1d-4ba4-83ae-957fdc8fd152 None admin] RBAC: Authorizing `identity:get_project(project_id=acd1ac1812eb4faeb54c342d511d42de)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:23.731168 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa86053-2b1d-4ba4-83ae-957fdc8fd152 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:23.733209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 183/369] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:32:23 2022] GET /identity/v3/projects/acd1ac1812eb4faeb54c342d511d42de => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:23.734055 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa86053-2b1d-4ba4-83ae-957fdc8fd152 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:23.749546 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c75df3-bee6-4891-938d-e83a6397381d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:23.750726 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c75df3-bee6-4891-938d-e83a6397381d None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:23.751022 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c75df3-bee6-4891-938d-e83a6397381d None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:23.751306 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c75df3-bee6-4891-938d-e83a6397381d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:23.764974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c75df3-bee6-4891-938d-e83a6397381d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:23.774780 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-d3c75df3-bee6-4891-938d-e83a6397381d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:32:23.774780 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:23.774780 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:23.774780 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:23.774780 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:23.774780 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:23.774780 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:23.774780 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:23.774780 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:23.774780 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:23.774780 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:23.774780 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:23.774780 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:23.774780 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:23.774780 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:23.774780 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:23.775682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:23.775682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:23.775682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:23.775682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:23.775682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:23.775682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:23.775682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:23.775682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:23.775682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:23.775682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:23.775682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:23.775682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:23.775682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:23.775682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:23.775682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:23.775682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:23.776938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:23.776938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:23.776938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:23.776938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:23.776938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:23.776938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:23.776938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:23.776938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:23.776938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:23.776938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:23.776938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:32:23.776938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:23.776938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 187/370] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:32:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:23.778376 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c75df3-bee6-4891-938d-e83a6397381d None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:23.782179 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:23.844310 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa86053-2b1d-4ba4-83ae-957fdc8fd152 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:23.877351 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:23.883668 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627bbcb8-0724-45a3-9e03-1b5b8ed26e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:23.884729 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bbcb8-0724-45a3-9e03-1b5b8ed26e4c None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:23.885384 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bbcb8-0724-45a3-9e03-1b5b8ed26e4c None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:23.885851 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bbcb8-0724-45a3-9e03-1b5b8ed26e4c None admin] PATH_INFO: `/v3/groups/eea5676f62f245f0b9d8d1594c15006c` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:23.888102 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627bbcb8-0724-45a3-9e03-1b5b8ed26e4c None admin] RBAC: Authorizing `identity:get_group(group_id=eea5676f62f245f0b9d8d1594c15006c)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:23.889715 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627bbcb8-0724-45a3-9e03-1b5b8ed26e4c None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:23.891469 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 184/371] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Jul 6 13:32:23 2022] GET /identity/v3/groups/eea5676f62f245f0b9d8d1594c15006c => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:23.892378 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627bbcb8-0724-45a3-9e03-1b5b8ed26e4c None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:23.905340 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c75df3-bee6-4891-938d-e83a6397381d None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:23.950192 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d717a7f8-8c19-47e8-9e6d-7e7594ed1790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:23.950863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717a7f8-8c19-47e8-9e6d-7e7594ed1790 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:23.951574 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717a7f8-8c19-47e8-9e6d-7e7594ed1790 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:23.951857 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717a7f8-8c19-47e8-9e6d-7e7594ed1790 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:23.952708 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d717a7f8-8c19-47e8-9e6d-7e7594ed1790 None admin] RBAC: Adding query filter params (role.id=65ecdcc5a9d0487ba529e54c626e152f, user.id=23388bbd8e964d41a17d6f736d6199c2, scope.system=all) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:23.953194 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d717a7f8-8c19-47e8-9e6d-7e7594ed1790 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:23.954778 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d717a7f8-8c19-47e8-9e6d-7e7594ed1790 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:23.974367 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 188/372] 10.222.0.20 () {56 vars in 1308 bytes} [Wed Jul 6 13:32:23 2022] GET /identity/v3/role_assignments?user.id=23388bbd8e964d41a17d6f736d6199c2&scope.system=all&role.id=65ecdcc5a9d0487ba529e54c626e152f => generated 510 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:23.975377 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d717a7f8-8c19-47e8-9e6d-7e7594ed1790 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:24.033657 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627bbcb8-0724-45a3-9e03-1b5b8ed26e4c None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:24.051866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:24.080315 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b9421-8976-48b9-a4f1-7462eefdc056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:24.081412 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b9421-8976-48b9-a4f1-7462eefdc056 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:24.081702 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b9421-8976-48b9-a4f1-7462eefdc056 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:24.082482 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b9421-8976-48b9-a4f1-7462eefdc056 None admin] PATH_INFO: `/v3/groups/eea5676f62f245f0b9d8d1594c15006c` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:24.084200 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b9421-8976-48b9-a4f1-7462eefdc056 None admin] RBAC: Authorizing `identity:get_group(group_id=eea5676f62f245f0b9d8d1594c15006c)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:24.085918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b9421-8976-48b9-a4f1-7462eefdc056 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:24.088368 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 185/373] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Jul 6 13:32:23 2022] GET /identity/v3/groups/eea5676f62f245f0b9d8d1594c15006c => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:24.089184 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b9421-8976-48b9-a4f1-7462eefdc056 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:24.103292 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d717a7f8-8c19-47e8-9e6d-7e7594ed1790 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:24.146788 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a319a-1d5d-4be1-923b-66bc5a067a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:24.147832 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a319a-1d5d-4be1-923b-66bc5a067a58 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:24.148263 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a319a-1d5d-4be1-923b-66bc5a067a58 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:24.148698 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a319a-1d5d-4be1-923b-66bc5a067a58 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:24.149576 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a319a-1d5d-4be1-923b-66bc5a067a58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:24.150107 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a319a-1d5d-4be1-923b-66bc5a067a58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:24.152363 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a319a-1d5d-4be1-923b-66bc5a067a58 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:24.169616 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 189/374] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:32:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:24.171037 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a319a-1d5d-4be1-923b-66bc5a067a58 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:24.178235 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:24.271037 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b9421-8976-48b9-a4f1-7462eefdc056 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:24.318570 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d27f72-030f-40b4-998d-522714b62beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:24.319716 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d27f72-030f-40b4-998d-522714b62beb None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:24.320202 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d27f72-030f-40b4-998d-522714b62beb None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:24.320684 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d27f72-030f-40b4-998d-522714b62beb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:24.321916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d27f72-030f-40b4-998d-522714b62beb None admin] RBAC: Adding query filter params (group.id=eea5676f62f245f0b9d8d1594c15006c, role.id=534400e7b28546a8a04c08f825c28566, scope.project.id=acd1ac1812eb4faeb54c342d511d42de) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:24.322540 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d27f72-030f-40b4-998d-522714b62beb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:24.324198 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d27f72-030f-40b4-998d-522714b62beb None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:24.326372 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a319a-1d5d-4be1-923b-66bc5a067a58 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:24.331935 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 186/375] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Jul 6 13:32:24 2022] GET /identity/v3/role_assignments?group.id=eea5676f62f245f0b9d8d1594c15006c&scope.project.id=acd1ac1812eb4faeb54c342d511d42de&role.id=534400e7b28546a8a04c08f825c28566 => generated 260 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:24.333038 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d27f72-030f-40b4-998d-522714b62beb None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:24.353389 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:24.368167 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4f28d-62e8-47c5-b5d4-c0f82697cdb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:24.373033 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4f28d-62e8-47c5-b5d4-c0f82697cdb0 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:24.373033 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4f28d-62e8-47c5-b5d4-c0f82697cdb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:24.373033 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4f28d-62e8-47c5-b5d4-c0f82697cdb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:24.373033 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a4f28d-62e8-47c5-b5d4-c0f82697cdb0 None admin] RBAC: Adding query filter params (group.id=eea5676f62f245f0b9d8d1594c15006c, role.id=b3244d1d9f004c1b8d3594fd30da1024, scope.project.id=acd1ac1812eb4faeb54c342d511d42de) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:24.373033 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a4f28d-62e8-47c5-b5d4-c0f82697cdb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:24.373698 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a4f28d-62e8-47c5-b5d4-c0f82697cdb0 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:24.382435 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 190/376] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Jul 6 13:32:24 2022] GET /identity/v3/role_assignments?group.id=eea5676f62f245f0b9d8d1594c15006c&scope.project.id=acd1ac1812eb4faeb54c342d511d42de&role.id=b3244d1d9f004c1b8d3594fd30da1024 => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:24.522114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d27f72-030f-40b4-998d-522714b62beb None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:24.565540 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:24.574155 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a056793-5c7b-4bc1-aebc-8e388daafc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:24.576507 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a056793-5c7b-4bc1-aebc-8e388daafc27 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:24.576796 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a056793-5c7b-4bc1-aebc-8e388daafc27 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:24.577099 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a056793-5c7b-4bc1-aebc-8e388daafc27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:24.640169 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf4d52-d782-48ae-8015-b76781391d8e None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:24.640511 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf4d52-d782-48ae-8015-b76781391d8e None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:24.640741 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf4d52-d782-48ae-8015-b76781391d8e None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:24.642500 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 191/377] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:24.648339 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f377c4-a725-4f19-b101-ae870a0c7bab None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:24.649021 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f377c4-a725-4f19-b101-ae870a0c7bab None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:24.649404 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f377c4-a725-4f19-b101-ae870a0c7bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:24.671307 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a056793-5c7b-4bc1-aebc-8e388daafc27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:24.672529 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a056793-5c7b-4bc1-aebc-8e388daafc27 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:24.676596 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-45f377c4-a725-4f19-b101-ae870a0c7bab None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:24.678155 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-45f377c4-a725-4f19-b101-ae870a0c7bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:24.771434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 192/378] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:24.827006 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f377c4-a725-4f19-b101-ae870a0c7bab None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:24.836754 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 187/379] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:32:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:32:24.844636 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a056793-5c7b-4bc1-aebc-8e388daafc27 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:24.875381 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:25.000248 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a056793-5c7b-4bc1-aebc-8e388daafc27 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:25.049309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f377c4-a725-4f19-b101-ae870a0c7bab None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:25.072405 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553508e7-83e2-4bbf-a0ac-7818224d4193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:25.073662 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553508e7-83e2-4bbf-a0ac-7818224d4193 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:25.074144 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553508e7-83e2-4bbf-a0ac-7818224d4193 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:25.075271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553508e7-83e2-4bbf-a0ac-7818224d4193 None admin] PATH_INFO: `/v3/projects/775434dcf39c43beaa4ed6a50089f763` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:25.078644 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:25.078776 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553508e7-83e2-4bbf-a0ac-7818224d4193 None admin] RBAC: Authorizing `identity:get_project(project_id=775434dcf39c43beaa4ed6a50089f763)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:25.080160 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553508e7-83e2-4bbf-a0ac-7818224d4193 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:25.081887 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 188/380] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:32:24 2022] GET /identity/v3/projects/775434dcf39c43beaa4ed6a50089f763 => generated 344 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:25.087522 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553508e7-83e2-4bbf-a0ac-7818224d4193 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:25.102708 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f70d19-770e-4874-9eee-32a5206a1ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:25.103524 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f70d19-770e-4874-9eee-32a5206a1ccc None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:25.103801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f70d19-770e-4874-9eee-32a5206a1ccc None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:25.104078 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f70d19-770e-4874-9eee-32a5206a1ccc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:25.118628 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f70d19-770e-4874-9eee-32a5206a1ccc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:25.130607 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-b8f70d19-770e-4874-9eee-32a5206a1ccc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 13:32:25.130607 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:25.130607 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:25.130607 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:25.130607 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:25.130607 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:25.130607 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:25.130607 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:25.130607 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:25.130607 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:25.130607 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:25.130607 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:25.130607 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:25.130607 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:25.130607 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:25.130607 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:25.131540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:25.131540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:25.131540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:25.131540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:25.131540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:25.131540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:25.131540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:25.131540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:25.131540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:25.131540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:25.131540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:25.131540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:25.131540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:25.131540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:25.131540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:25.131540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:25.132336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:25.132336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:25.132336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:25.132336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:25.132336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:25.132336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:25.132336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:25.132336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:25.132336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:25.132336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:25.132336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 13:32:25.132336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:25.135656 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 193/381] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jul 6 13:32:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:32:25.139170 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f70d19-770e-4874-9eee-32a5206a1ccc None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:25.243880 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:25.254277 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553508e7-83e2-4bbf-a0ac-7818224d4193 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:25.332461 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253fcc2-d917-4165-990f-3877a0ec7170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:25.333273 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253fcc2-d917-4165-990f-3877a0ec7170 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:25.333559 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253fcc2-d917-4165-990f-3877a0ec7170 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:25.333835 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253fcc2-d917-4165-990f-3877a0ec7170 None admin] PATH_INFO: `/v3/groups/38d7a68fb87448fd80851500f93422a2` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:25.345514 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9253fcc2-d917-4165-990f-3877a0ec7170 None admin] RBAC: Authorizing `identity:get_group(group_id=38d7a68fb87448fd80851500f93422a2)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:25.346976 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9253fcc2-d917-4165-990f-3877a0ec7170 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:25.348685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 189/382] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Jul 6 13:32:25 2022] GET /identity/v3/groups/38d7a68fb87448fd80851500f93422a2 => generated 229 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:25.357658 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f70d19-770e-4874-9eee-32a5206a1ccc None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:25.359173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253fcc2-d917-4165-990f-3877a0ec7170 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:25.429956 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0426380-39f0-4a45-bd3b-cc74110dc383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:25.430628 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0426380-39f0-4a45-bd3b-cc74110dc383 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:25.430910 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0426380-39f0-4a45-bd3b-cc74110dc383 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:25.431185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0426380-39f0-4a45-bd3b-cc74110dc383 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:25.431943 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0426380-39f0-4a45-bd3b-cc74110dc383 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:25.432250 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0426380-39f0-4a45-bd3b-cc74110dc383 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:25.433696 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0426380-39f0-4a45-bd3b-cc74110dc383 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:25.454649 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 194/383] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:32:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:25.467529 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242ac92-8d97-461d-9876-760189a6a902 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:25.467839 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242ac92-8d97-461d-9876-760189a6a902 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:25.468095 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242ac92-8d97-461d-9876-760189a6a902 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:25.469913 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 195/384] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:25.470630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242ac92-8d97-461d-9876-760189a6a902 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:25.529739 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:25.531873 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253fcc2-d917-4165-990f-3877a0ec7170 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:25.567685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf30e24-d9ed-4464-b3ff-13b24e7b0297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:25.568344 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf30e24-d9ed-4464-b3ff-13b24e7b0297 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:25.568625 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf30e24-d9ed-4464-b3ff-13b24e7b0297 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:25.569197 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf30e24-d9ed-4464-b3ff-13b24e7b0297 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:25.569796 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf30e24-d9ed-4464-b3ff-13b24e7b0297 None admin] RBAC: Adding query filter params (group.id=38d7a68fb87448fd80851500f93422a2, role.id=795daa2c78354fd48e4a741e1182cc98, scope.project.id=775434dcf39c43beaa4ed6a50089f763) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:25.570111 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf30e24-d9ed-4464-b3ff-13b24e7b0297 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:25.571728 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf30e24-d9ed-4464-b3ff-13b24e7b0297 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:25.578899 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 190/385] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Jul 6 13:32:25 2022] GET /identity/v3/role_assignments?group.id=38d7a68fb87448fd80851500f93422a2&scope.project.id=775434dcf39c43beaa4ed6a50089f763&role.id=795daa2c78354fd48e4a741e1182cc98 => generated 260 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:25.580150 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15accc8a-8f93-4cc5-90dd-7565a47a22bf None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:25.580440 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15accc8a-8f93-4cc5-90dd-7565a47a22bf None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:25.580718 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15accc8a-8f93-4cc5-90dd-7565a47a22bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:25.597985 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-15accc8a-8f93-4cc5-90dd-7565a47a22bf None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:25.599130 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-15accc8a-8f93-4cc5-90dd-7565a47a22bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:25.601030 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242ac92-8d97-461d-9876-760189a6a902 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:25.618212 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:25.651126 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b304055-3182-4ceb-9891-8ea92ad9031e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:25.652022 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b304055-3182-4ceb-9891-8ea92ad9031e None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:25.652310 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b304055-3182-4ceb-9891-8ea92ad9031e None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:25.653187 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b304055-3182-4ceb-9891-8ea92ad9031e None admin] PATH_INFO: `/v3/groups/eea5676f62f245f0b9d8d1594c15006c` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:25.655247 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b304055-3182-4ceb-9891-8ea92ad9031e None admin] RBAC: Authorizing `identity:get_group(group_id=eea5676f62f245f0b9d8d1594c15006c)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:25.656701 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b304055-3182-4ceb-9891-8ea92ad9031e None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:25.658679 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 196/386] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Jul 6 13:32:25 2022] GET /identity/v3/groups/eea5676f62f245f0b9d8d1594c15006c => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:25.664319 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b304055-3182-4ceb-9891-8ea92ad9031e None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:25.686462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 191/387] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:25.714282 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15accc8a-8f93-4cc5-90dd-7565a47a22bf None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:25.789030 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b304055-3182-4ceb-9891-8ea92ad9031e None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:25.796480 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:25.835618 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b4a5a3-f2a0-4b24-b46d-72051ea2b298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:25.836567 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4a5a3-f2a0-4b24-b46d-72051ea2b298 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:25.836853 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4a5a3-f2a0-4b24-b46d-72051ea2b298 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:25.837157 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4a5a3-f2a0-4b24-b46d-72051ea2b298 None admin] PATH_INFO: `/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:25.838433 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b4a5a3-f2a0-4b24-b46d-72051ea2b298 None admin] RBAC: Authorizing `identity:get_project(project_id=202d7ba5c7f34872aa3ab82e5d2b2f93)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:25.840052 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b4a5a3-f2a0-4b24-b46d-72051ea2b298 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:25.842721 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 197/388] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:32:25 2022] GET /identity/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:25.848652 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b4a5a3-f2a0-4b24-b46d-72051ea2b298 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:25.887235 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15accc8a-8f93-4cc5-90dd-7565a47a22bf None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:25.973403 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb05cde-f210-442c-86b5-29b446307b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:25.974211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb05cde-f210-442c-86b5-29b446307b45 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:25.974487 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb05cde-f210-442c-86b5-29b446307b45 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:25.974759 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb05cde-f210-442c-86b5-29b446307b45 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:25.998055 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb05cde-f210-442c-86b5-29b446307b45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:26.005854 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-8bb05cde-f210-442c-86b5-29b446307b45 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:32:26.005854 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:26.005854 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:26.005854 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:26.005854 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:26.005854 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:26.005854 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:26.005854 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:26.005854 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:26.005854 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:26.005854 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:26.005854 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:26.005854 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:26.005854 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:26.005854 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:26.005854 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:26.006743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:26.006743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:26.006743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:26.006743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:26.006743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:26.006743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:26.006743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:26.006743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:26.006743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:26.006743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:26.006743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:26.006743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:26.006743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:26.006743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:26.006743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:26.006743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:26.007541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:26.007541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:26.007541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:26.007541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:26.007541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:26.007541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:26.007541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:26.007541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:26.007541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:26.007541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:26.007541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:32:26.007541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:26.009592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b4a5a3-f2a0-4b24-b46d-72051ea2b298 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:26.010982 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 192/389] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:32:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:32:26.021680 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb05cde-f210-442c-86b5-29b446307b45 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:26.051013 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:26.086320 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6ad98e-4147-4db8-b601-b8a104f97bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:26.087230 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ad98e-4147-4db8-b601-b8a104f97bf0 None admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:26.087509 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ad98e-4147-4db8-b601-b8a104f97bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:26.087784 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ad98e-4147-4db8-b601-b8a104f97bf0 None admin] PATH_INFO: `/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93/groups/eea5676f62f245f0b9d8d1594c15006c/roles/534400e7b28546a8a04c08f825c28566` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:26.097959 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6ad98e-4147-4db8-b601-b8a104f97bf0 None admin] RBAC: Authorizing `identity:create_grant(project_id=202d7ba5c7f34872aa3ab82e5d2b2f93, group_id=eea5676f62f245f0b9d8d1594c15006c, role_id=534400e7b28546a8a04c08f825c28566)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:26.099504 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6ad98e-4147-4db8-b601-b8a104f97bf0 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:26.140355 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 198/390] 10.222.0.20 () {58 vars in 1442 bytes} [Wed Jul 6 13:32:25 2022] PUT /identity/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93/groups/eea5676f62f245f0b9d8d1594c15006c/roles/534400e7b28546a8a04c08f825c28566 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:32:26.189234 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb05cde-f210-442c-86b5-29b446307b45 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:26.251145 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef4a78-36d3-4358-8ea2-7ba394dd537f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:26.251840 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef4a78-36d3-4358-8ea2-7ba394dd537f None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:26.252127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef4a78-36d3-4358-8ea2-7ba394dd537f None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:26.252404 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef4a78-36d3-4358-8ea2-7ba394dd537f None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:26.253235 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef4a78-36d3-4358-8ea2-7ba394dd537f None admin] RBAC: Adding query filter params (name=member) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:26.253550 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef4a78-36d3-4358-8ea2-7ba394dd537f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:26.254934 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef4a78-36d3-4358-8ea2-7ba394dd537f None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:26.267814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 193/391] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jul 6 13:32:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:26.274161 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6ad98e-4147-4db8-b601-b8a104f97bf0 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:26.352756 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:26.447864 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6ad98e-4147-4db8-b601-b8a104f97bf0 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:26.486631 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ac53a-14cf-4550-9166-24745224c52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:26.487547 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ac53a-14cf-4550-9166-24745224c52c None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:26.487892 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ac53a-14cf-4550-9166-24745224c52c None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:26.488225 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ac53a-14cf-4550-9166-24745224c52c None admin] PATH_INFO: `/v3/groups/eea5676f62f245f0b9d8d1594c15006c` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:26.489689 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09ac53a-14cf-4550-9166-24745224c52c None admin] RBAC: Authorizing `identity:get_group(group_id=eea5676f62f245f0b9d8d1594c15006c)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:26.491139 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09ac53a-14cf-4550-9166-24745224c52c None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:26.492779 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 199/392] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Jul 6 13:32:26 2022] GET /identity/v3/groups/eea5676f62f245f0b9d8d1594c15006c => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:26.500349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef4a78-36d3-4358-8ea2-7ba394dd537f None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:26.583142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:26.613092 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef4a78-36d3-4358-8ea2-7ba394dd537f None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:26.653131 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aba71b-ed5d-4762-9347-819ece7911ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:26.654019 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aba71b-ed5d-4762-9347-819ece7911ac None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:26.654304 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aba71b-ed5d-4762-9347-819ece7911ac None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:26.654582 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aba71b-ed5d-4762-9347-819ece7911ac None admin] PATH_INFO: `/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:26.661755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aba71b-ed5d-4762-9347-819ece7911ac None admin] RBAC: Authorizing `identity:get_project(project_id=202d7ba5c7f34872aa3ab82e5d2b2f93)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:26.663215 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aba71b-ed5d-4762-9347-819ece7911ac None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:26.665135 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 194/393] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:32:26 2022] GET /identity/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:26.671328 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ac53a-14cf-4550-9166-24745224c52c None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:26.797759 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ac53a-14cf-4550-9166-24745224c52c None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:26.801296 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:26.841167 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1acb7e6-1cb9-456a-826a-4733a582467e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:26.842407 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1acb7e6-1cb9-456a-826a-4733a582467e None admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:26.842941 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1acb7e6-1cb9-456a-826a-4733a582467e None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:26.843427 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1acb7e6-1cb9-456a-826a-4733a582467e None admin] PATH_INFO: `/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93/groups/eea5676f62f245f0b9d8d1594c15006c/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:26.849849 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1acb7e6-1cb9-456a-826a-4733a582467e None admin] RBAC: Authorizing `identity:create_grant(project_id=202d7ba5c7f34872aa3ab82e5d2b2f93, group_id=eea5676f62f245f0b9d8d1594c15006c, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:26.851587 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1acb7e6-1cb9-456a-826a-4733a582467e None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:26.879593 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 200/394] 10.222.0.20 () {58 vars in 1442 bytes} [Wed Jul 6 13:32:26 2022] PUT /identity/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93/groups/eea5676f62f245f0b9d8d1594c15006c/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:32:27.029890 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68355c-9ed6-440a-b336-989e5e5924c9 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:27.030232 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68355c-9ed6-440a-b336-989e5e5924c9 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:27.030498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68355c-9ed6-440a-b336-989e5e5924c9 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:27.032261 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 195/395] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:27.037677 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4562e-ef69-48c5-8207-a39e8c8c9aee None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:27.037965 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4562e-ef69-48c5-8207-a39e8c8c9aee None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:27.038242 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4562e-ef69-48c5-8207-a39e8c8c9aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:27.060154 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-5bc4562e-ef69-48c5-8207-a39e8c8c9aee None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:27.061373 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-5bc4562e-ef69-48c5-8207-a39e8c8c9aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:27.156896 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 201/396] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:27.179400 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd68355c-9ed6-440a-b336-989e5e5924c9 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:27.328836 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd68355c-9ed6-440a-b336-989e5e5924c9 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:27.370777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20001c19-d9f5-4409-b9a7-e661200ef076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:27.371638 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20001c19-d9f5-4409-b9a7-e661200ef076 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:27.371922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20001c19-d9f5-4409-b9a7-e661200ef076 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:27.372187 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20001c19-d9f5-4409-b9a7-e661200ef076 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:27.387348 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20001c19-d9f5-4409-b9a7-e661200ef076 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:27.395247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-20001c19-d9f5-4409-b9a7-e661200ef076 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:32:27.395247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:27.395247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:27.395247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:27.395247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:27.395247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:27.395247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:27.395247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:27.395247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:27.395247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:27.395247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:27.395247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:27.395247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:27.395247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:27.395247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:27.395247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:27.396131 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:27.396131 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:27.396131 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:27.396131 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:27.396131 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:27.396131 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:27.396131 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:27.396131 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:27.396131 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:27.396131 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:27.396131 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:27.396131 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:27.396131 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:27.396131 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:27.396131 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:27.396131 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:27.397041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:27.397041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:27.397041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:27.397041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:27.397041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:27.397041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:27.397041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:27.397041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:27.397041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:27.397041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:27.397041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:32:27.397041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:27.397041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 196/397] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:32:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:27.403262 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4562e-ef69-48c5-8207-a39e8c8c9aee None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:27.440772 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc2097-5a20-4c26-b76b-aa03721f6b90 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:27.441392 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc2097-5a20-4c26-b76b-aa03721f6b90 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:27.441845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc2097-5a20-4c26-b76b-aa03721f6b90 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:27.443622 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 197/398] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:32:27.448970 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ce9c4-4a7e-414a-93a5-a699aeb89f29 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:27.450076 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ce9c4-4a7e-414a-93a5-a699aeb89f29 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:27.450711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ce9c4-4a7e-414a-93a5-a699aeb89f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:27.457229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:27.471347 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-156ce9c4-4a7e-414a-93a5-a699aeb89f29 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:27.472790 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-156ce9c4-4a7e-414a-93a5-a699aeb89f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:27.572005 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:27.583396 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4562e-ef69-48c5-8207-a39e8c8c9aee None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:27.596336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 198/399] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:27.624602 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156ce9c4-4a7e-414a-93a5-a699aeb89f29 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:27.659026 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f8559e-b589-4511-bc0b-7f0c2d23048b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:27.660025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f8559e-b589-4511-bc0b-7f0c2d23048b None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:27.660473 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f8559e-b589-4511-bc0b-7f0c2d23048b None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:27.660935 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f8559e-b589-4511-bc0b-7f0c2d23048b None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:27.662679 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f8559e-b589-4511-bc0b-7f0c2d23048b None admin] RBAC: Adding query filter params (name=member) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:27.663804 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f8559e-b589-4511-bc0b-7f0c2d23048b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:27.665435 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f8559e-b589-4511-bc0b-7f0c2d23048b None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:27.683034 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 202/400] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jul 6 13:32:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:27.691021 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f8559e-b589-4511-bc0b-7f0c2d23048b None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:27.802117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156ce9c4-4a7e-414a-93a5-a699aeb89f29 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:27.860497 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48625309-01b9-4133-95b5-7b19abddcb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:27.863379 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48625309-01b9-4133-95b5-7b19abddcb53 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:27.863845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48625309-01b9-4133-95b5-7b19abddcb53 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:27.865650 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48625309-01b9-4133-95b5-7b19abddcb53 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:27.869507 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:27.884520 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f8559e-b589-4511-bc0b-7f0c2d23048b None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:27.889752 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48625309-01b9-4133-95b5-7b19abddcb53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:27.908871 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-48625309-01b9-4133-95b5-7b19abddcb53 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 13:32:27.908871 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:27.908871 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:27.908871 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:27.908871 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:27.908871 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:27.908871 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:27.908871 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:27.908871 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:27.908871 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:27.908871 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:27.908871 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:27.908871 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:27.908871 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:27.908871 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:27.908871 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:27.909865 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:27.909865 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:27.909865 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:27.909865 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:27.909865 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:27.909865 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:27.909865 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:27.909865 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:27.909865 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:27.909865 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:27.909865 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:27.909865 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:27.909865 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:27.909865 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:27.909865 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:27.909865 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:27.910665 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:27.910665 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:27.910665 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:27.910665 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:27.910665 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:27.910665 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:27.910665 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:27.910665 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:27.910665 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:27.910665 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:27.910665 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 13:32:27.910665 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:27.913793 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 199/401] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jul 6 13:32:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:32:27.917711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48625309-01b9-4133-95b5-7b19abddcb53 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:27.943254 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88961d39-f321-468a-9d23-f3eb5b423678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:27.944222 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88961d39-f321-468a-9d23-f3eb5b423678 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:27.944510 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88961d39-f321-468a-9d23-f3eb5b423678 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:27.944786 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88961d39-f321-468a-9d23-f3eb5b423678 None admin] PATH_INFO: `/v3/groups/eea5676f62f245f0b9d8d1594c15006c` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:27.946077 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88961d39-f321-468a-9d23-f3eb5b423678 None admin] RBAC: Authorizing `identity:get_group(group_id=eea5676f62f245f0b9d8d1594c15006c)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:27.947590 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88961d39-f321-468a-9d23-f3eb5b423678 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:27.951807 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 203/402] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Jul 6 13:32:27 2022] GET /identity/v3/groups/eea5676f62f245f0b9d8d1594c15006c => generated 226 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:27.960396 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88961d39-f321-468a-9d23-f3eb5b423678 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:27.998668 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:28.111435 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48625309-01b9-4133-95b5-7b19abddcb53 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:28.127689 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88961d39-f321-468a-9d23-f3eb5b423678 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:28.155484 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ceb74-a1b1-4574-b16e-01d49842c09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:28.156393 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ceb74-a1b1-4574-b16e-01d49842c09d None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:28.157236 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ceb74-a1b1-4574-b16e-01d49842c09d None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:28.157658 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ceb74-a1b1-4574-b16e-01d49842c09d None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:28.158650 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ceb74-a1b1-4574-b16e-01d49842c09d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:28.159099 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ceb74-a1b1-4574-b16e-01d49842c09d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:28.163366 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ceb74-a1b1-4574-b16e-01d49842c09d None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:28.176020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 200/403] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:32:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:28.184312 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ceb74-a1b1-4574-b16e-01d49842c09d None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:28.188111 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bed4be-dc58-439b-bb48-c2868ceff8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:28.188995 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bed4be-dc58-439b-bb48-c2868ceff8e0 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:28.189307 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bed4be-dc58-439b-bb48-c2868ceff8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:28.189584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bed4be-dc58-439b-bb48-c2868ceff8e0 None admin] PATH_INFO: `/v3/projects/acd1ac1812eb4faeb54c342d511d42de` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:28.192616 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bed4be-dc58-439b-bb48-c2868ceff8e0 None admin] RBAC: Authorizing `identity:get_project(project_id=acd1ac1812eb4faeb54c342d511d42de)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:28.194150 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bed4be-dc58-439b-bb48-c2868ceff8e0 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:28.195797 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 204/404] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:32:27 2022] GET /identity/v3/projects/acd1ac1812eb4faeb54c342d511d42de => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:28.203141 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bed4be-dc58-439b-bb48-c2868ceff8e0 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:28.314259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:28.361209 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:28.363661 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ceb74-a1b1-4574-b16e-01d49842c09d None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:28.370855 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bed4be-dc58-439b-bb48-c2868ceff8e0 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:28.413855 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db0d00-ac17-4799-8f67-b9e5c094e829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:28.414915 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35abdcd4-d4cc-419f-a72d-33262fe22fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:28.415291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db0d00-ac17-4799-8f67-b9e5c094e829 None admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:28.415671 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db0d00-ac17-4799-8f67-b9e5c094e829 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:28.416046 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db0d00-ac17-4799-8f67-b9e5c094e829 None admin] PATH_INFO: `/v3/projects/acd1ac1812eb4faeb54c342d511d42de/groups/eea5676f62f245f0b9d8d1594c15006c/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:28.416459 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abdcd4-d4cc-419f-a72d-33262fe22fbf None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:28.416883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abdcd4-d4cc-419f-a72d-33262fe22fbf None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:28.417329 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abdcd4-d4cc-419f-a72d-33262fe22fbf None admin] PATH_INFO: `/v3/groups/eea5676f62f245f0b9d8d1594c15006c` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:28.419440 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db0d00-ac17-4799-8f67-b9e5c094e829 None admin] RBAC: Authorizing `identity:create_grant(project_id=acd1ac1812eb4faeb54c342d511d42de, group_id=eea5676f62f245f0b9d8d1594c15006c, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:28.420036 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35abdcd4-d4cc-419f-a72d-33262fe22fbf None admin] RBAC: Authorizing `identity:get_group(group_id=eea5676f62f245f0b9d8d1594c15006c)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:28.421105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db0d00-ac17-4799-8f67-b9e5c094e829 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:28.421962 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35abdcd4-d4cc-419f-a72d-33262fe22fbf None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:28.424390 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 201/405] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Jul 6 13:32:28 2022] GET /identity/v3/groups/eea5676f62f245f0b9d8d1594c15006c => generated 226 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:28.426584 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1c5d1-099e-4ffb-a210-812c3c16bf99 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:28.426925 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1c5d1-099e-4ffb-a210-812c3c16bf99 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:28.427207 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1c5d1-099e-4ffb-a210-812c3c16bf99 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:28.433078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 202/406] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:28.433078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f1c5d1-099e-4ffb-a210-812c3c16bf99 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:28.445992 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 205/407] 10.222.0.20 () {58 vars in 1442 bytes} [Wed Jul 6 13:32:28 2022] PUT /identity/v3/projects/acd1ac1812eb4faeb54c342d511d42de/groups/eea5676f62f245f0b9d8d1594c15006c/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:32:28.445992 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f33a8f-12a0-4ff1-87cb-2d36d20106cd None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:28.445992 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f33a8f-12a0-4ff1-87cb-2d36d20106cd None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:28.445992 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f33a8f-12a0-4ff1-87cb-2d36d20106cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:28.461566 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-f8f33a8f-12a0-4ff1-87cb-2d36d20106cd None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:28.462979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-f8f33a8f-12a0-4ff1-87cb-2d36d20106cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:28.579411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 206/408] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:28.582073 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f1c5d1-099e-4ffb-a210-812c3c16bf99 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:28.624752 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:28.648236 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0cb28-ed0d-41e8-9409-55bb17b8e289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:28.649582 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0cb28-ed0d-41e8-9409-55bb17b8e289 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:28.650073 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0cb28-ed0d-41e8-9409-55bb17b8e289 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:28.650552 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0cb28-ed0d-41e8-9409-55bb17b8e289 None admin] PATH_INFO: `/v3/projects/acd1ac1812eb4faeb54c342d511d42de` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:28.654744 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c0cb28-ed0d-41e8-9409-55bb17b8e289 None admin] RBAC: Authorizing `identity:get_project(project_id=acd1ac1812eb4faeb54c342d511d42de)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:28.656503 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c0cb28-ed0d-41e8-9409-55bb17b8e289 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:28.658914 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 203/409] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:32:28 2022] GET /identity/v3/projects/acd1ac1812eb4faeb54c342d511d42de => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:28.660563 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f33a8f-12a0-4ff1-87cb-2d36d20106cd None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:28.664202 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0cb28-ed0d-41e8-9409-55bb17b8e289 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:28.786571 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:28.809461 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0cb28-ed0d-41e8-9409-55bb17b8e289 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:28.849055 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce49a9b-11e2-4555-ab55-611a7873e0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:28.850051 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce49a9b-11e2-4555-ab55-611a7873e0c6 None admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:28.850329 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce49a9b-11e2-4555-ab55-611a7873e0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:28.850602 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce49a9b-11e2-4555-ab55-611a7873e0c6 None admin] PATH_INFO: `/v3/projects/acd1ac1812eb4faeb54c342d511d42de/groups/eea5676f62f245f0b9d8d1594c15006c/roles/534400e7b28546a8a04c08f825c28566` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:28.853036 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce49a9b-11e2-4555-ab55-611a7873e0c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=acd1ac1812eb4faeb54c342d511d42de, group_id=eea5676f62f245f0b9d8d1594c15006c, role_id=534400e7b28546a8a04c08f825c28566)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:28.854908 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce49a9b-11e2-4555-ab55-611a7873e0c6 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:28.863020 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f33a8f-12a0-4ff1-87cb-2d36d20106cd None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:28.879623 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 204/410] 10.222.0.20 () {58 vars in 1442 bytes} [Wed Jul 6 13:32:28 2022] PUT /identity/v3/projects/acd1ac1812eb4faeb54c342d511d42de/groups/eea5676f62f245f0b9d8d1594c15006c/roles/534400e7b28546a8a04c08f825c28566 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:32:28.912655 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a48d30-ce85-469f-901a-6452f7efb102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:28.913517 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a48d30-ce85-469f-901a-6452f7efb102 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:28.913797 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a48d30-ce85-469f-901a-6452f7efb102 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:28.914072 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a48d30-ce85-469f-901a-6452f7efb102 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:28.927341 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a48d30-ce85-469f-901a-6452f7efb102 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:28.936065 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-c0a48d30-ce85-469f-901a-6452f7efb102 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:32:28.936065 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:28.936065 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:28.936065 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:28.936065 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:28.936065 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:28.936065 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:28.936065 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:28.936065 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:28.936065 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:28.936065 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:28.936065 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:28.936065 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:28.936065 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:28.936065 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:28.936065 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:28.937083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:28.937083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:28.937083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:28.937083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:28.937083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:28.937083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:28.937083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:28.937083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:28.937083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:28.937083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:28.937083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:28.937083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:28.937083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:28.937083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:28.937083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:28.937083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:28.938423 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:28.938423 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:28.938423 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:28.938423 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:28.938423 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:28.938423 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:28.938423 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:28.938423 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:28.938423 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:28.938423 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:28.938423 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:32:28.938423 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:28.938423 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 207/411] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:32:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:32:28.943736 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce49a9b-11e2-4555-ab55-611a7873e0c6 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:29.001907 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:29.063233 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce49a9b-11e2-4555-ab55-611a7873e0c6 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:29.098478 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72163c-b7e9-45b1-84ea-464b46c46e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:29.099157 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72163c-b7e9-45b1-84ea-464b46c46e83 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:29.099438 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72163c-b7e9-45b1-84ea-464b46c46e83 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:29.099771 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72163c-b7e9-45b1-84ea-464b46c46e83 None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:29.100530 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca72163c-b7e9-45b1-84ea-464b46c46e83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:29.100885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca72163c-b7e9-45b1-84ea-464b46c46e83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:29.102300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca72163c-b7e9-45b1-84ea-464b46c46e83 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:29.111679 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee01fd-f935-4399-b24d-3bfade9038b9 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:29.112348 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee01fd-f935-4399-b24d-3bfade9038b9 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:29.112578 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 205/412] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:32:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:29.113107 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee01fd-f935-4399-b24d-3bfade9038b9 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:29.116730 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 208/413] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:32:29.118948 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72163c-b7e9-45b1-84ea-464b46c46e83 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:29.123679 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a472a1-d5a3-4082-9fc2-1a7970bf651f None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:29.124231 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a472a1-d5a3-4082-9fc2-1a7970bf651f None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:29.124741 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a472a1-d5a3-4082-9fc2-1a7970bf651f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:29.146583 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-67a472a1-d5a3-4082-9fc2-1a7970bf651f None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:29.148104 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-67a472a1-d5a3-4082-9fc2-1a7970bf651f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:29.225206 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 209/414] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:29.250076 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a472a1-d5a3-4082-9fc2-1a7970bf651f None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:29.260826 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72163c-b7e9-45b1-84ea-464b46c46e83 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:29.303085 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:29.348081 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287bc1ea-0f69-4b72-8f94-12bc4bc3be59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:29.349273 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287bc1ea-0f69-4b72-8f94-12bc4bc3be59 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:29.352820 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287bc1ea-0f69-4b72-8f94-12bc4bc3be59 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:29.353959 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287bc1ea-0f69-4b72-8f94-12bc4bc3be59 None admin] PATH_INFO: `/v3/groups/38d7a68fb87448fd80851500f93422a2` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:29.357340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287bc1ea-0f69-4b72-8f94-12bc4bc3be59 None admin] RBAC: Authorizing `identity:get_group(group_id=38d7a68fb87448fd80851500f93422a2)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:29.359039 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287bc1ea-0f69-4b72-8f94-12bc4bc3be59 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:29.362358 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 206/415] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Jul 6 13:32:29 2022] GET /identity/v3/groups/38d7a68fb87448fd80851500f93422a2 => generated 229 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:29.367774 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287bc1ea-0f69-4b72-8f94-12bc4bc3be59 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:29.391836 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:29.434576 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a472a1-d5a3-4082-9fc2-1a7970bf651f None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:29.501458 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f233d-d63d-46d4-b84e-015295b174a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:29.502268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f233d-d63d-46d4-b84e-015295b174a5 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:29.502546 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f233d-d63d-46d4-b84e-015295b174a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:29.502818 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f233d-d63d-46d4-b84e-015295b174a5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:29.514008 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f233d-d63d-46d4-b84e-015295b174a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:29.522115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-606f233d-d63d-46d4-b84e-015295b174a5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 13:32:29.522115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:29.522115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:29.522115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:29.522115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:29.522115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:29.522115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:29.522115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:29.522115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:29.522115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:29.522115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:29.522115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:29.522115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:29.522115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:29.522115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:29.522115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:29.523002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:29.523002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:29.523002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:29.523002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:29.523002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:29.523002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:29.523002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:29.523002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:29.523002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:29.523002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:29.523002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:29.523002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:29.523002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:29.523002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:29.523002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:29.523002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:29.523809 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:29.523809 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:29.523809 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:29.523809 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:29.523809 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:29.523809 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:29.523809 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:29.523809 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:29.523809 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:29.523809 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:29.523809 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 13:32:29.523809 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:29.523809 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 210/416] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jul 6 13:32:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:32:29.524688 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736dce60-d066-4f55-9c1a-b6cc504a4296 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:29.524807 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736dce60-d066-4f55-9c1a-b6cc504a4296 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:29.525109 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736dce60-d066-4f55-9c1a-b6cc504a4296 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:29.526619 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 211/417] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:29.531668 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bff52f-15e0-480e-a9e1-9b9cc267344e None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:29.531956 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bff52f-15e0-480e-a9e1-9b9cc267344e None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:29.532234 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bff52f-15e0-480e-a9e1-9b9cc267344e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:29.539035 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287bc1ea-0f69-4b72-8f94-12bc4bc3be59 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:29.549316 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-63bff52f-15e0-480e-a9e1-9b9cc267344e None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:29.550888 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-63bff52f-15e0-480e-a9e1-9b9cc267344e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:29.577462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215a002-899d-45ed-9bdf-2dda7f5d458e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:29.578811 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215a002-899d-45ed-9bdf-2dda7f5d458e None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:29.579325 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215a002-899d-45ed-9bdf-2dda7f5d458e None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:29.579801 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215a002-899d-45ed-9bdf-2dda7f5d458e None admin] PATH_INFO: `/v3/projects/775434dcf39c43beaa4ed6a50089f763` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:29.581277 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215a002-899d-45ed-9bdf-2dda7f5d458e None admin] RBAC: Authorizing `identity:get_project(project_id=775434dcf39c43beaa4ed6a50089f763)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:29.583075 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215a002-899d-45ed-9bdf-2dda7f5d458e None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:29.585570 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 207/418] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:32:29 2022] GET /identity/v3/projects/775434dcf39c43beaa4ed6a50089f763 => generated 344 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:29.586637 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215a002-899d-45ed-9bdf-2dda7f5d458e None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:29.633435 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 212/419] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 13:32:29.634205 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bff52f-15e0-480e-a9e1-9b9cc267344e None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:29.650782 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:29.711995 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215a002-899d-45ed-9bdf-2dda7f5d458e None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:29.721242 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:29.756783 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e639b84-019a-4e51-b2f6-4413d7c92669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:29.757615 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e639b84-019a-4e51-b2f6-4413d7c92669 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:29.757907 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e639b84-019a-4e51-b2f6-4413d7c92669 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:29.758188 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e639b84-019a-4e51-b2f6-4413d7c92669 None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:29.759140 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e639b84-019a-4e51-b2f6-4413d7c92669 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:29.759457 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e639b84-019a-4e51-b2f6-4413d7c92669 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:29.760836 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e639b84-019a-4e51-b2f6-4413d7c92669 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:29.769704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bff52f-15e0-480e-a9e1-9b9cc267344e None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:29.774900 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 208/420] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:32:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:29.775803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e639b84-019a-4e51-b2f6-4413d7c92669 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:29.812965 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0fbc6-4596-45e8-a1b1-58c49f1f00eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:29.813924 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0fbc6-4596-45e8-a1b1-58c49f1f00eb None admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:29.814210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0fbc6-4596-45e8-a1b1-58c49f1f00eb None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:29.814537 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0fbc6-4596-45e8-a1b1-58c49f1f00eb None admin] PATH_INFO: `/v3/projects/775434dcf39c43beaa4ed6a50089f763/groups/38d7a68fb87448fd80851500f93422a2/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:29.817688 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e0fbc6-4596-45e8-a1b1-58c49f1f00eb None admin] RBAC: Authorizing `identity:create_grant(project_id=775434dcf39c43beaa4ed6a50089f763, group_id=38d7a68fb87448fd80851500f93422a2, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:29.819112 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e0fbc6-4596-45e8-a1b1-58c49f1f00eb None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:29.837094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 213/421] 10.222.0.20 () {58 vars in 1442 bytes} [Wed Jul 6 13:32:29 2022] PUT /identity/v3/projects/775434dcf39c43beaa4ed6a50089f763/groups/38d7a68fb87448fd80851500f93422a2/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:32:29.838017 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0fbc6-4596-45e8-a1b1-58c49f1f00eb None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:29.987184 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e639b84-019a-4e51-b2f6-4413d7c92669 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:30.024175 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:30.081815 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a296f-31fc-42e9-a8f2-4a9b96a2ca21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:30.083217 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a296f-31fc-42e9-a8f2-4a9b96a2ca21 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:30.083517 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a296f-31fc-42e9-a8f2-4a9b96a2ca21 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:30.083800 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a296f-31fc-42e9-a8f2-4a9b96a2ca21 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:30.091866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0fbc6-4596-45e8-a1b1-58c49f1f00eb None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:30.106162 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55a296f-31fc-42e9-a8f2-4a9b96a2ca21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:30.126035 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:30.126214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-e55a296f-31fc-42e9-a8f2-4a9b96a2ca21 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:32:30.126214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:30.126214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:30.126214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:30.126214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:30.126214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:30.126214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:30.126214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:30.126214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:30.126214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:30.126214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:30.126214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:30.126214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:30.126214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:30.126214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:30.126214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:30.127057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:30.127057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:30.127057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:30.127057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:30.127057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:30.127057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:30.127057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:30.127057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:30.127057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:30.127057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:30.127057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:30.127057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:30.127057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:30.127057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:30.127057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:30.127057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:30.131932 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:30.131932 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:30.131932 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:30.131932 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:30.131932 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:30.131932 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:30.131932 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:30.131932 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:30.131932 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:30.131932 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:30.131932 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:32:30.131932 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:30.131932 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 209/422] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:32:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:30.149021 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a296f-31fc-42e9-a8f2-4a9b96a2ca21 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:30.153174 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35502877-ef7d-41fa-88c7-0767ad9bcdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:30.153843 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35502877-ef7d-41fa-88c7-0767ad9bcdf2 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:30.154284 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35502877-ef7d-41fa-88c7-0767ad9bcdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:30.154573 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35502877-ef7d-41fa-88c7-0767ad9bcdf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:30.333033 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35502877-ef7d-41fa-88c7-0767ad9bcdf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:30.333033 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35502877-ef7d-41fa-88c7-0767ad9bcdf2 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:30.416250 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a296f-31fc-42e9-a8f2-4a9b96a2ca21 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:30.461350 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5eb27-7b99-4b8d-b0bf-b9fe8718badf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:30.462407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5eb27-7b99-4b8d-b0bf-b9fe8718badf None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:30.462898 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5eb27-7b99-4b8d-b0bf-b9fe8718badf None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:30.463377 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5eb27-7b99-4b8d-b0bf-b9fe8718badf None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:30.464330 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca5eb27-7b99-4b8d-b0bf-b9fe8718badf None admin] RBAC: Adding query filter params (name=member) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:30.465402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca5eb27-7b99-4b8d-b0bf-b9fe8718badf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:30.467597 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca5eb27-7b99-4b8d-b0bf-b9fe8718badf None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:30.482131 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 210/423] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jul 6 13:32:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:30.483639 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5eb27-7b99-4b8d-b0bf-b9fe8718badf None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:30.518752 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:30.599117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5eb27-7b99-4b8d-b0bf-b9fe8718badf None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:30.638046 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e6460-5a36-4327-81e2-87a2d64937b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:30.639085 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e6460-5a36-4327-81e2-87a2d64937b9 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:30.639579 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e6460-5a36-4327-81e2-87a2d64937b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:30.640527 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e6460-5a36-4327-81e2-87a2d64937b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:30.701107 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 214/424] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:32:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 858 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:32:30.714850 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35502877-ef7d-41fa-88c7-0767ad9bcdf2 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:30.733405 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e6460-5a36-4327-81e2-87a2d64937b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:30.735575 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e6460-5a36-4327-81e2-87a2d64937b9 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:30.770983 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:30.794878 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:30.861056 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 211/425] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:32:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:32:30.862798 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e6460-5a36-4327-81e2-87a2d64937b9 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:30.991824 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35502877-ef7d-41fa-88c7-0767ad9bcdf2 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:31.001799 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e6460-5a36-4327-81e2-87a2d64937b9 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:31.036978 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669c063-2220-49fe-8ba2-4007cf50d10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:31.038904 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669c063-2220-49fe-8ba2-4007cf50d10b None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:31.039375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669c063-2220-49fe-8ba2-4007cf50d10b None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:31.039838 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669c063-2220-49fe-8ba2-4007cf50d10b None admin] PATH_INFO: `/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:31.042033 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669c063-2220-49fe-8ba2-4007cf50d10b None admin] RBAC: Authorizing `identity:get_project(project_id=202d7ba5c7f34872aa3ab82e5d2b2f93)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:31.043692 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669c063-2220-49fe-8ba2-4007cf50d10b None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:31.045443 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:31.047534 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 215/426] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:32:30 2022] GET /identity/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93 => generated 298 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:31.049237 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbd541-550f-4f35-acec-f0b4ae61ab26 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:31.049749 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbd541-550f-4f35-acec-f0b4ae61ab26 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:31.050221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbd541-550f-4f35-acec-f0b4ae61ab26 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:31.055811 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 216/427] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:31.057284 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bbd541-550f-4f35-acec-f0b4ae61ab26 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:31.062378 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3d3f01-ba24-42fd-91b3-2bc3ce553c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:31.064232 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d3f01-ba24-42fd-91b3-2bc3ce553c78 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:31.064590 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d3f01-ba24-42fd-91b3-2bc3ce553c78 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:31.064941 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d3f01-ba24-42fd-91b3-2bc3ce553c78 None admin] PATH_INFO: `/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:31.067359 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3d3f01-ba24-42fd-91b3-2bc3ce553c78 None admin] RBAC: Authorizing `identity:get_project(project_id=202d7ba5c7f34872aa3ab82e5d2b2f93)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:31.069893 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3d3f01-ba24-42fd-91b3-2bc3ce553c78 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:31.077590 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 212/428] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:32:30 2022] GET /identity/v3/projects/202d7ba5c7f34872aa3ab82e5d2b2f93 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:31.078977 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec3d54-26f9-415f-8c6d-56eb1931fbfe None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:31.078977 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec3d54-26f9-415f-8c6d-56eb1931fbfe None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:31.079256 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec3d54-26f9-415f-8c6d-56eb1931fbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:31.101948 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-1fec3d54-26f9-415f-8c6d-56eb1931fbfe None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:31.103989 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-1fec3d54-26f9-415f-8c6d-56eb1931fbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:31.182214 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:31.197078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 213/429] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:31.198606 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec3d54-26f9-415f-8c6d-56eb1931fbfe None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:31.202685 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bbd541-550f-4f35-acec-f0b4ae61ab26 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:31.248863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b3c8a-823a-438e-b4e0-e7c3572cc536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:31.249768 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b3c8a-823a-438e-b4e0-e7c3572cc536 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:31.250055 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b3c8a-823a-438e-b4e0-e7c3572cc536 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:31.250332 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b3c8a-823a-438e-b4e0-e7c3572cc536 None admin] PATH_INFO: `/v3/groups/eea5676f62f245f0b9d8d1594c15006c` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:31.251528 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898b3c8a-823a-438e-b4e0-e7c3572cc536 None admin] RBAC: Authorizing `identity:get_group(group_id=eea5676f62f245f0b9d8d1594c15006c)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:31.252948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898b3c8a-823a-438e-b4e0-e7c3572cc536 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:31.255462 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 217/430] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Jul 6 13:32:31 2022] GET /identity/v3/groups/eea5676f62f245f0b9d8d1594c15006c => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:31.256388 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b3c8a-823a-438e-b4e0-e7c3572cc536 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:31.332354 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec3d54-26f9-415f-8c6d-56eb1931fbfe None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:31.361620 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:31.368923 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de13749-fec3-49d1-9e5f-65dd2ee53396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:31.369839 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de13749-fec3-49d1-9e5f-65dd2ee53396 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:31.370284 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de13749-fec3-49d1-9e5f-65dd2ee53396 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:31.370733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de13749-fec3-49d1-9e5f-65dd2ee53396 None admin] PATH_INFO: `/v3/groups/eea5676f62f245f0b9d8d1594c15006c` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:31.372603 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de13749-fec3-49d1-9e5f-65dd2ee53396 None admin] RBAC: Authorizing `identity:get_group(group_id=eea5676f62f245f0b9d8d1594c15006c)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:31.373641 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b3c8a-823a-438e-b4e0-e7c3572cc536 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:31.374114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de13749-fec3-49d1-9e5f-65dd2ee53396 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:31.375827 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 214/431] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Jul 6 13:32:31 2022] GET /identity/v3/groups/eea5676f62f245f0b9d8d1594c15006c => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:31.377303 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e22cba-2a8b-4369-b431-082e2afed2d6 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:31.377723 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e22cba-2a8b-4369-b431-082e2afed2d6 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:31.378162 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e22cba-2a8b-4369-b431-082e2afed2d6 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:31.379619 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 215/432] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:31.380483 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e22cba-2a8b-4369-b431-082e2afed2d6 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:31.418794 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75583b38-f3a3-4967-8af3-99f9688436b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:31.421332 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75583b38-f3a3-4967-8af3-99f9688436b9 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:31.421790 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75583b38-f3a3-4967-8af3-99f9688436b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:31.422207 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75583b38-f3a3-4967-8af3-99f9688436b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:31.436795 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75583b38-f3a3-4967-8af3-99f9688436b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:31.447634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-75583b38-f3a3-4967-8af3-99f9688436b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:32:31.447634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:31.447634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:31.447634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:31.447634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:31.447634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:31.447634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:31.447634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:31.447634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:31.447634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:31.447634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:31.447634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:31.447634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:31.447634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:31.447634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:31.447634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:31.448676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:31.448676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:31.448676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:31.448676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:31.448676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:31.448676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:31.448676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:31.448676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:31.448676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:31.448676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:31.448676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:31.448676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:31.448676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:31.448676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:31.448676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:31.448676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:31.449516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:31.449516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:31.449516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:31.449516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:31.449516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:31.449516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:31.449516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:31.449516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:31.449516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:31.449516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:31.449516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 13:32:31.449516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:31.449516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 218/433] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:32:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:31.450691 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75583b38-f3a3-4967-8af3-99f9688436b9 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:31.471898 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:31.496910 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e22cba-2a8b-4369-b431-082e2afed2d6 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:31.534215 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef319b7d-8bd6-427b-8f81-0c23cdaba35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:31.534940 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef319b7d-8bd6-427b-8f81-0c23cdaba35e None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:31.535225 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef319b7d-8bd6-427b-8f81-0c23cdaba35e None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:31.535503 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef319b7d-8bd6-427b-8f81-0c23cdaba35e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:31.536362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef319b7d-8bd6-427b-8f81-0c23cdaba35e None admin] RBAC: Adding query filter params (group.id=eea5676f62f245f0b9d8d1594c15006c, role.id=534400e7b28546a8a04c08f825c28566, scope.project.id=202d7ba5c7f34872aa3ab82e5d2b2f93) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:31.536682 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef319b7d-8bd6-427b-8f81-0c23cdaba35e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:31.538155 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef319b7d-8bd6-427b-8f81-0c23cdaba35e None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:31.544375 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 216/434] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Jul 6 13:32:31 2022] GET /identity/v3/role_assignments?group.id=eea5676f62f245f0b9d8d1594c15006c&scope.project.id=202d7ba5c7f34872aa3ab82e5d2b2f93&role.id=534400e7b28546a8a04c08f825c28566 => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:31.545475 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612031e9-6af2-4437-ba92-dab83f9f678a None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:31.545766 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612031e9-6af2-4437-ba92-dab83f9f678a None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:31.546048 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612031e9-6af2-4437-ba92-dab83f9f678a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:31.564399 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-612031e9-6af2-4437-ba92-dab83f9f678a None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:31.565913 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-612031e9-6af2-4437-ba92-dab83f9f678a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:31.573171 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75583b38-f3a3-4967-8af3-99f9688436b9 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:31.625610 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c0533-377e-4679-809a-9802f2a17b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:31.626280 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c0533-377e-4679-809a-9802f2a17b56 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:31.626566 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c0533-377e-4679-809a-9802f2a17b56 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:31.626844 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c0533-377e-4679-809a-9802f2a17b56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:31.627693 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02c0533-377e-4679-809a-9802f2a17b56 None admin] RBAC: Adding query filter params (group.id=eea5676f62f245f0b9d8d1594c15006c, role.id=b3244d1d9f004c1b8d3594fd30da1024, scope.project.id=202d7ba5c7f34872aa3ab82e5d2b2f93) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:31.628008 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02c0533-377e-4679-809a-9802f2a17b56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:31.629551 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02c0533-377e-4679-809a-9802f2a17b56 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:31.636579 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 219/435] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Jul 6 13:32:31 2022] GET /identity/v3/role_assignments?group.id=eea5676f62f245f0b9d8d1594c15006c&scope.project.id=202d7ba5c7f34872aa3ab82e5d2b2f93&role.id=b3244d1d9f004c1b8d3594fd30da1024 => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:31.637828 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c0533-377e-4679-809a-9802f2a17b56 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:31.639623 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:31.661026 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 217/436] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:31.680652 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612031e9-6af2-4437-ba92-dab83f9f678a None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:31.741147 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:31.765454 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c0533-377e-4679-809a-9802f2a17b56 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:31.810104 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c780a32e-0c80-4386-b46f-1714f38ec6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:31.811163 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780a32e-0c80-4386-b46f-1714f38ec6c0 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:31.812837 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780a32e-0c80-4386-b46f-1714f38ec6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:31.813462 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780a32e-0c80-4386-b46f-1714f38ec6c0 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:31.814527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c780a32e-0c80-4386-b46f-1714f38ec6c0 None admin] RBAC: Adding query filter params (name=member) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:31.815049 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c780a32e-0c80-4386-b46f-1714f38ec6c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:31.816649 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c780a32e-0c80-4386-b46f-1714f38ec6c0 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:31.826906 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 220/437] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jul 6 13:32:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:31.833512 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c780a32e-0c80-4386-b46f-1714f38ec6c0 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:31.836197 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612031e9-6af2-4437-ba92-dab83f9f678a None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:31.887308 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0adec81-d32b-4a1a-8015-5d4616ba0aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:31.888503 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0adec81-d32b-4a1a-8015-5d4616ba0aa3 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:31.888984 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0adec81-d32b-4a1a-8015-5d4616ba0aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:31.889287 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0adec81-d32b-4a1a-8015-5d4616ba0aa3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:31.906260 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0adec81-d32b-4a1a-8015-5d4616ba0aa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:31.914706 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-d0adec81-d32b-4a1a-8015-5d4616ba0aa3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 13:32:31.914706 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:31.914706 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:31.914706 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:31.914706 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:31.914706 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:31.914706 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:31.914706 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:31.914706 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:31.914706 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:31.914706 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:31.914706 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:31.914706 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:31.914706 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:31.914706 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:31.914706 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:31.915613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:31.915613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:31.915613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:31.915613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:31.915613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:31.915613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:31.915613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:31.915613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:31.915613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:31.915613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:31.915613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:31.915613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:31.915613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:31.915613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:31.915613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:31.915613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:31.916419 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:31.916419 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:31.916419 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:31.916419 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:31.916419 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:31.916419 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:31.916419 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:31.916419 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:31.916419 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:31.916419 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:31.916419 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 13:32:31.916419 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:31.917641 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 218/438] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jul 6 13:32:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:32:31.923450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0adec81-d32b-4a1a-8015-5d4616ba0aa3 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:31.970447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c780a32e-0c80-4386-b46f-1714f38ec6c0 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:31.997126 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:32.017227 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7191b-d0c5-489d-a986-e17c8b447904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:32.017227 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7191b-d0c5-489d-a986-e17c8b447904 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:32.017227 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7191b-d0c5-489d-a986-e17c8b447904 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:32.017227 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7191b-d0c5-489d-a986-e17c8b447904 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:32.032926 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:32.061675 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0adec81-d32b-4a1a-8015-5d4616ba0aa3 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:32.103163 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb7191b-d0c5-489d-a986-e17c8b447904 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:32.105184 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb7191b-d0c5-489d-a986-e17c8b447904 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:32.108942 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee881a2b-04ba-4cb6-9fe6-b1f83fb9a2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:32.109650 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee881a2b-04ba-4cb6-9fe6-b1f83fb9a2ab None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:32.109939 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee881a2b-04ba-4cb6-9fe6-b1f83fb9a2ab None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:32.110218 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee881a2b-04ba-4cb6-9fe6-b1f83fb9a2ab None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:32.111004 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee881a2b-04ba-4cb6-9fe6-b1f83fb9a2ab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:32.111320 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee881a2b-04ba-4cb6-9fe6-b1f83fb9a2ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:32.112683 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee881a2b-04ba-4cb6-9fe6-b1f83fb9a2ab None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:32.127428 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 219/439] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:32:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:32.132920 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee881a2b-04ba-4cb6-9fe6-b1f83fb9a2ab None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:32.229164 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 221/440] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:32:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:32:32.235619 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7191b-d0c5-489d-a986-e17c8b447904 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:32.261982 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee881a2b-04ba-4cb6-9fe6-b1f83fb9a2ab None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:32.276415 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:32.303614 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c4555e-bf3f-402a-b7f2-63842cf8c963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:32.304096 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:32.304266 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c4555e-bf3f-402a-b7f2-63842cf8c963 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:32.304767 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c4555e-bf3f-402a-b7f2-63842cf8c963 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:32.305267 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c4555e-bf3f-402a-b7f2-63842cf8c963 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:32.366467 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7191b-d0c5-489d-a986-e17c8b447904 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:32.392643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c4555e-bf3f-402a-b7f2-63842cf8c963 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:32.394150 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c4555e-bf3f-402a-b7f2-63842cf8c963 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:32.406704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511dcdf-a0c7-4355-ae27-ad3fdb6b0846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:32.407574 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511dcdf-a0c7-4355-ae27-ad3fdb6b0846 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:32.408454 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511dcdf-a0c7-4355-ae27-ad3fdb6b0846 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:32.408454 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511dcdf-a0c7-4355-ae27-ad3fdb6b0846 None admin] PATH_INFO: `/v3/projects/acd1ac1812eb4faeb54c342d511d42de` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:32.409614 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f511dcdf-a0c7-4355-ae27-ad3fdb6b0846 None admin] RBAC: Authorizing `identity:get_project(project_id=acd1ac1812eb4faeb54c342d511d42de)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:32.410997 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f511dcdf-a0c7-4355-ae27-ad3fdb6b0846 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:32.413302 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 222/441] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:32:32 2022] GET /identity/v3/projects/acd1ac1812eb4faeb54c342d511d42de => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:32.414940 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb11101-18cd-4149-9e11-60c022f2e8db None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:32.415234 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb11101-18cd-4149-9e11-60c022f2e8db None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:32.415513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb11101-18cd-4149-9e11-60c022f2e8db None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:32.416767 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 223/442] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:32.422853 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb11101-18cd-4149-9e11-60c022f2e8db None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:32.504377 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 220/443] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:32:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 06 13:32:32.505684 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe7df1-0968-4373-8869-f139d2ce4e78 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:32.505979 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe7df1-0968-4373-8869-f139d2ce4e78 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:32.506259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe7df1-0968-4373-8869-f139d2ce4e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:32.521712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-ecfe7df1-0968-4373-8869-f139d2ce4e78 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:32.522827 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-ecfe7df1-0968-4373-8869-f139d2ce4e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:32.532388 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:32.532920 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb11101-18cd-4149-9e11-60c022f2e8db None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:32.547034 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:32.568835 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876dd14-143c-4dbc-8879-56f4eb5301d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:32.569711 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876dd14-143c-4dbc-8879-56f4eb5301d1 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:32.569992 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876dd14-143c-4dbc-8879-56f4eb5301d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:32.570265 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876dd14-143c-4dbc-8879-56f4eb5301d1 None admin] PATH_INFO: `/v3/groups/eea5676f62f245f0b9d8d1594c15006c` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:32.571413 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876dd14-143c-4dbc-8879-56f4eb5301d1 None admin] RBAC: Authorizing `identity:get_group(group_id=eea5676f62f245f0b9d8d1594c15006c)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:32.572823 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876dd14-143c-4dbc-8879-56f4eb5301d1 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:32.574494 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 224/444] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Jul 6 13:32:32 2022] GET /identity/v3/groups/eea5676f62f245f0b9d8d1594c15006c => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:32.575324 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876dd14-143c-4dbc-8879-56f4eb5301d1 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:32.592371 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 221/445] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:32.593312 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe7df1-0968-4373-8869-f139d2ce4e78 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:32.673215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876dd14-143c-4dbc-8879-56f4eb5301d1 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:32.701318 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe7df1-0968-4373-8869-f139d2ce4e78 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:32.707771 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225d3829-c433-4496-8fab-3eb6c5c23cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:32.709058 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d3829-c433-4496-8fab-3eb6c5c23cd5 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:32.709343 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d3829-c433-4496-8fab-3eb6c5c23cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:32.709629 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d3829-c433-4496-8fab-3eb6c5c23cd5 None admin] PATH_INFO: `/v3/projects/acd1ac1812eb4faeb54c342d511d42de` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:32.711154 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225d3829-c433-4496-8fab-3eb6c5c23cd5 None admin] RBAC: Authorizing `identity:get_project(project_id=acd1ac1812eb4faeb54c342d511d42de)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:32.712913 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225d3829-c433-4496-8fab-3eb6c5c23cd5 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:32.715545 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 225/446] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:32:32 2022] GET /identity/v3/projects/acd1ac1812eb4faeb54c342d511d42de => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:32.716663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225d3829-c433-4496-8fab-3eb6c5c23cd5 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:32.736089 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485bdc96-d185-495b-8f7a-cfc450b6c2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:32.736755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bdc96-d185-495b-8f7a-cfc450b6c2a5 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:32.737077 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bdc96-d185-495b-8f7a-cfc450b6c2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:32.737363 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bdc96-d185-495b-8f7a-cfc450b6c2a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:32.738195 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485bdc96-d185-495b-8f7a-cfc450b6c2a5 None admin] RBAC: Adding query filter params (group.id=eea5676f62f245f0b9d8d1594c15006c, role.id=b3244d1d9f004c1b8d3594fd30da1024, scope.project.id=acd1ac1812eb4faeb54c342d511d42de) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:32.738511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485bdc96-d185-495b-8f7a-cfc450b6c2a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:32.739933 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485bdc96-d185-495b-8f7a-cfc450b6c2a5 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:32.746775 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 222/447] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Jul 6 13:32:32 2022] GET /identity/v3/role_assignments?group.id=eea5676f62f245f0b9d8d1594c15006c&scope.project.id=acd1ac1812eb4faeb54c342d511d42de&role.id=b3244d1d9f004c1b8d3594fd30da1024 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:32.747569 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485bdc96-d185-495b-8f7a-cfc450b6c2a5 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:32.759329 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:32.767440 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:32.826419 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225d3829-c433-4496-8fab-3eb6c5c23cd5 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:32.863601 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc129874-2592-41fb-8954-3f6db620c9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:32.864414 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc129874-2592-41fb-8954-3f6db620c9a7 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:32.864695 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc129874-2592-41fb-8954-3f6db620c9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:32.865031 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc129874-2592-41fb-8954-3f6db620c9a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:32.867414 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485bdc96-d185-495b-8f7a-cfc450b6c2a5 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:32.877912 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc129874-2592-41fb-8954-3f6db620c9a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:32.885957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-fc129874-2592-41fb-8954-3f6db620c9a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:32:32.885957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:32.885957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:32.885957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:32.885957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:32.885957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:32.885957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:32.885957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:32.885957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:32.885957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:32.885957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:32.885957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:32.885957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:32.885957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:32.885957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:32.885957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:32.887513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:32.887513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:32.887513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:32.887513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:32.887513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:32.887513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:32.887513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:32.887513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:32.887513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:32.887513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:32.887513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:32.887513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:32.887513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:32.887513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:32.887513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:32.887513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:32.888312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:32.888312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:32.888312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:32.888312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:32.888312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:32.888312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:32.888312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:32.888312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:32.888312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:32.888312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:32.888312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:32:32.888312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:32.888312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 226/448] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:32:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:32.894105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc129874-2592-41fb-8954-3f6db620c9a7 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:32.913513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950402c-1a01-4555-9cdd-6bc383d20c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:32.914393 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950402c-1a01-4555-9cdd-6bc383d20c36 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:32.914664 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950402c-1a01-4555-9cdd-6bc383d20c36 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:32.914938 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950402c-1a01-4555-9cdd-6bc383d20c36 None admin] PATH_INFO: `/v3/groups/eea5676f62f245f0b9d8d1594c15006c` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:32.917137 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7950402c-1a01-4555-9cdd-6bc383d20c36 None admin] RBAC: Authorizing `identity:get_group(group_id=eea5676f62f245f0b9d8d1594c15006c)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:32.920009 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7950402c-1a01-4555-9cdd-6bc383d20c36 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:32.926021 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 223/449] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Jul 6 13:32:32 2022] GET /identity/v3/groups/eea5676f62f245f0b9d8d1594c15006c => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:32.927354 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950402c-1a01-4555-9cdd-6bc383d20c36 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:33.018273 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:33.018927 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc129874-2592-41fb-8954-3f6db620c9a7 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:33.031036 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:33.048786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950402c-1a01-4555-9cdd-6bc383d20c36 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:33.068226 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15944c4-167b-4704-bf21-b19ad8519c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:33.068897 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15944c4-167b-4704-bf21-b19ad8519c03 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:33.069208 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15944c4-167b-4704-bf21-b19ad8519c03 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:33.069484 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15944c4-167b-4704-bf21-b19ad8519c03 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:33.070235 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15944c4-167b-4704-bf21-b19ad8519c03 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:33.070544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15944c4-167b-4704-bf21-b19ad8519c03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:33.071911 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15944c4-167b-4704-bf21-b19ad8519c03 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:33.082214 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 227/450] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:32:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:33.087393 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad69f0a-8158-43a4-be7d-2ed74a76b37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:33.087994 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15944c4-167b-4704-bf21-b19ad8519c03 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:33.088110 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad69f0a-8158-43a4-be7d-2ed74a76b37a None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:33.088331 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad69f0a-8158-43a4-be7d-2ed74a76b37a None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:33.088611 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad69f0a-8158-43a4-be7d-2ed74a76b37a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:33.089511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad69f0a-8158-43a4-be7d-2ed74a76b37a None admin] RBAC: Adding query filter params (group.id=eea5676f62f245f0b9d8d1594c15006c, role.id=534400e7b28546a8a04c08f825c28566, scope.project.id=acd1ac1812eb4faeb54c342d511d42de) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:33.090034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad69f0a-8158-43a4-be7d-2ed74a76b37a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:33.091602 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad69f0a-8158-43a4-be7d-2ed74a76b37a None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:33.098721 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 224/451] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Jul 6 13:32:32 2022] GET /identity/v3/role_assignments?group.id=eea5676f62f245f0b9d8d1594c15006c&scope.project.id=acd1ac1812eb4faeb54c342d511d42de&role.id=534400e7b28546a8a04c08f825c28566 => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:33.195895 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15944c4-167b-4704-bf21-b19ad8519c03 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:33.230063 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5688a893-0934-448c-859a-8652aec45183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:33.231068 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688a893-0934-448c-859a-8652aec45183 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:33.231357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688a893-0934-448c-859a-8652aec45183 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:33.231633 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688a893-0934-448c-859a-8652aec45183 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:33.271811 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:33.306693 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5688a893-0934-448c-859a-8652aec45183 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:33.308135 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5688a893-0934-448c-859a-8652aec45183 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:33.409022 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 228/452] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:32:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 06 13:32:33.414571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad69f0a-8158-43a4-be7d-2ed74a76b37a None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:33.514644 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad69f0a-8158-43a4-be7d-2ed74a76b37a None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:33.546769 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba4592-9341-4676-9302-1e230608b2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:33.547626 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba4592-9341-4676-9302-1e230608b2a8 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:33.547914 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba4592-9341-4676-9302-1e230608b2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:33.548193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba4592-9341-4676-9302-1e230608b2a8 None admin] PATH_INFO: `/v3/projects/775434dcf39c43beaa4ed6a50089f763` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:33.549651 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddba4592-9341-4676-9302-1e230608b2a8 None admin] RBAC: Authorizing `identity:get_project(project_id=775434dcf39c43beaa4ed6a50089f763)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:33.551098 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddba4592-9341-4676-9302-1e230608b2a8 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:33.552798 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 225/453] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:32:33 2022] GET /identity/v3/projects/775434dcf39c43beaa4ed6a50089f763 => generated 344 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:33.558323 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5688a893-0934-448c-859a-8652aec45183 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:33.624907 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:33.654905 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5688a893-0934-448c-859a-8652aec45183 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:33.687740 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d4ca68-9852-4f3e-9f59-350fcb8d55ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:33.688509 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4ca68-9852-4f3e-9f59-350fcb8d55ee None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:33.688792 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4ca68-9852-4f3e-9f59-350fcb8d55ee None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:33.689097 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4ca68-9852-4f3e-9f59-350fcb8d55ee None admin] PATH_INFO: `/v3/groups/38d7a68fb87448fd80851500f93422a2` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:33.690240 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d4ca68-9852-4f3e-9f59-350fcb8d55ee None admin] RBAC: Authorizing `identity:get_group(group_id=38d7a68fb87448fd80851500f93422a2)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:33.691661 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d4ca68-9852-4f3e-9f59-350fcb8d55ee None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:33.693284 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 229/454] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Jul 6 13:32:33 2022] GET /identity/v3/groups/38d7a68fb87448fd80851500f93422a2 => generated 229 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:33.698913 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba4592-9341-4676-9302-1e230608b2a8 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:33.710118 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:33.799629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba4592-9341-4676-9302-1e230608b2a8 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:33.844413 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67673641-36cd-47a6-b78a-e96bacd664e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:33.845086 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67673641-36cd-47a6-b78a-e96bacd664e1 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:33.845371 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67673641-36cd-47a6-b78a-e96bacd664e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:33.845647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67673641-36cd-47a6-b78a-e96bacd664e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:33.846471 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67673641-36cd-47a6-b78a-e96bacd664e1 None admin] RBAC: Adding query filter params (group.id=38d7a68fb87448fd80851500f93422a2, role.id=795daa2c78354fd48e4a741e1182cc98, scope.project.id=775434dcf39c43beaa4ed6a50089f763) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:33.846787 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67673641-36cd-47a6-b78a-e96bacd664e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:33.848201 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67673641-36cd-47a6-b78a-e96bacd664e1 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:33.853856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 226/455] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Jul 6 13:32:33 2022] GET /identity/v3/role_assignments?group.id=38d7a68fb87448fd80851500f93422a2&scope.project.id=775434dcf39c43beaa4ed6a50089f763&role.id=795daa2c78354fd48e4a741e1182cc98 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:36.612334 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe896772-3074-4d18-9939-1d9d905b2ef6 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:36.613411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe896772-3074-4d18-9939-1d9d905b2ef6 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:36.613915 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe896772-3074-4d18-9939-1d9d905b2ef6 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:36.615487 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 230/456] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:36.620597 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92b450-45ab-4146-bf51-c90416d00875 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:36.622231 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92b450-45ab-4146-bf51-c90416d00875 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:36.622730 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92b450-45ab-4146-bf51-c90416d00875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:36.646043 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-5b92b450-45ab-4146-bf51-c90416d00875 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:36.647595 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-5b92b450-45ab-4146-bf51-c90416d00875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:36.675187 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ceebc-3e82-42b4-8682-b483f8e457c8 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:36.675813 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ceebc-3e82-42b4-8682-b483f8e457c8 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:36.676321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ceebc-3e82-42b4-8682-b483f8e457c8 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:36.678152 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 231/457] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:36.684115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a6426-f4c0-4c7b-ae7e-a6e828d17fe3 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:36.684735 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a6426-f4c0-4c7b-ae7e-a6e828d17fe3 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:36.685242 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a6426-f4c0-4c7b-ae7e-a6e828d17fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:36.702470 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-4b8a6426-f4c0-4c7b-ae7e-a6e828d17fe3 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:36.703972 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-4b8a6426-f4c0-4c7b-ae7e-a6e828d17fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:36.725507 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 227/458] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:36.756348 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c85a94-51bb-49f1-8e09-587708802efd None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:36.756970 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c85a94-51bb-49f1-8e09-587708802efd None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:36.757500 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c85a94-51bb-49f1-8e09-587708802efd None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:36.759229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 228/459] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:32:36.764309 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40fa0b-b8d2-4c21-99c4-009d649ed039 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:36.764848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40fa0b-b8d2-4c21-99c4-009d649ed039 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:36.765391 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40fa0b-b8d2-4c21-99c4-009d649ed039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:36.777824 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:36.783782 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-ac40fa0b-b8d2-4c21-99c4-009d649ed039 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:36.785212 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-ac40fa0b-b8d2-4c21-99c4-009d649ed039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:36.792155 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 232/460] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 13:32:36.797144 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a6426-f4c0-4c7b-ae7e-a6e828d17fe3 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:36.873731 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 229/461] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 13:32:36.875047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40fa0b-b8d2-4c21-99c4-009d649ed039 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:36.875554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:36.922206 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a6426-f4c0-4c7b-ae7e-a6e828d17fe3 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:36.957235 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47dd2a-8fd3-45d0-a32f-e49773c8aafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:36.958028 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47dd2a-8fd3-45d0-a32f-e49773c8aafc None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:36.958308 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47dd2a-8fd3-45d0-a32f-e49773c8aafc None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:36.958582 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47dd2a-8fd3-45d0-a32f-e49773c8aafc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:36.969384 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f47dd2a-8fd3-45d0-a32f-e49773c8aafc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:36.970835 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f47dd2a-8fd3-45d0-a32f-e49773c8aafc None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:36.983890 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40fa0b-b8d2-4c21-99c4-009d649ed039 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:36.988527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-0f47dd2a-8fd3-45d0-a32f-e49773c8aafc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:36.988527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:36.988527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:36.988527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:36.988527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:36.988527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:36.988527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:36.988527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:36.988527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:36.988527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:36.988527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:36.988527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:36.988527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:36.988527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:36.988527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:36.989554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:36.989554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:36.989554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:36.989554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:36.989554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:36.989554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:36.989554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:36.989554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:36.989554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:36.989554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:36.989554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:36.989554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:36.989554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:36.989554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:36.989554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:36.989554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:36.989554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:36.991019 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:36.991019 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:36.991019 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:36.991019 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:36.991019 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 233/462] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:36.991019 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47dd2a-8fd3-45d0-a32f-e49773c8aafc None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:37.023769 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:37.024891 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a920b3a-de6e-4286-82a2-3c471d9d8c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:37.026082 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a920b3a-de6e-4286-82a2-3c471d9d8c39 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:37.027623 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a920b3a-de6e-4286-82a2-3c471d9d8c39 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:37.027623 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a920b3a-de6e-4286-82a2-3c471d9d8c39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:37.037947 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a920b3a-de6e-4286-82a2-3c471d9d8c39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:37.039393 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a920b3a-de6e-4286-82a2-3c471d9d8c39 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:37.058955 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-9a920b3a-de6e-4286-82a2-3c471d9d8c39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:37.058955 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:37.058955 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:37.058955 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:37.058955 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:37.058955 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:37.058955 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:37.058955 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:37.058955 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:37.058955 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:37.058955 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:37.058955 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:37.058955 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:37.058955 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:37.058955 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:37.059814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:37.059814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:37.059814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:37.059814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:37.059814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:37.059814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:37.059814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:37.059814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:37.059814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:37.059814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:37.059814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:37.059814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:37.059814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:37.059814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:37.059814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:37.059814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:37.059814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:37.060672 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:37.060672 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:37.060672 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:37.060672 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:37.060672 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 230/463] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:37.060940 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a920b3a-de6e-4286-82a2-3c471d9d8c39 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:37.096776 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47dd2a-8fd3-45d0-a32f-e49773c8aafc None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:37.116765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:37.130947 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbed083-476d-4406-a86b-b8fb069cbe54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:37.131984 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbed083-476d-4406-a86b-b8fb069cbe54 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:37.132464 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbed083-476d-4406-a86b-b8fb069cbe54 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:37.132917 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbed083-476d-4406-a86b-b8fb069cbe54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:37.143643 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbed083-476d-4406-a86b-b8fb069cbe54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:37.145337 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbed083-476d-4406-a86b-b8fb069cbe54 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:37.163967 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-2cbed083-476d-4406-a86b-b8fb069cbe54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:37.163967 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:37.163967 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:37.163967 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:37.163967 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:37.163967 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:37.163967 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:37.163967 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:37.163967 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:37.163967 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:37.163967 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:37.163967 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:37.163967 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:37.163967 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:37.163967 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:37.164914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:37.164914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:37.164914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:37.164914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:37.164914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:37.164914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:37.164914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:37.164914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:37.164914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:37.164914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:37.164914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:37.164914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:37.164914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:37.164914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:37.164914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:37.164914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:37.164914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:37.165999 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:37.165999 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:37.165999 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:37.165999 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:37.174687 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 234/464] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:37.174687 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49f8f9-a60a-4712-9a2c-cd661f3721ed None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:37.174687 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49f8f9-a60a-4712-9a2c-cd661f3721ed None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:37.174687 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49f8f9-a60a-4712-9a2c-cd661f3721ed None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:37.174687 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 235/465] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:37.174687 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49f8f9-a60a-4712-9a2c-cd661f3721ed None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:37.187830 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a920b3a-de6e-4286-82a2-3c471d9d8c39 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:37.227555 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0e0a4-1ff6-416b-93ee-9368b5d9425d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:37.228217 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0e0a4-1ff6-416b-93ee-9368b5d9425d None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:37.228495 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0e0a4-1ff6-416b-93ee-9368b5d9425d None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:37.228769 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0e0a4-1ff6-416b-93ee-9368b5d9425d None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:37.229523 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf0e0a4-1ff6-416b-93ee-9368b5d9425d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:37.229829 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf0e0a4-1ff6-416b-93ee-9368b5d9425d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:37.231571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf0e0a4-1ff6-416b-93ee-9368b5d9425d None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:37.248493 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 231/466] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:37.248493 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0e0a4-1ff6-416b-93ee-9368b5d9425d None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:37.288391 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49f8f9-a60a-4712-9a2c-cd661f3721ed None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:37.296811 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:37.332625 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b508ad20-fad3-49db-bb02-52d0462ea41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:37.333320 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b508ad20-fad3-49db-bb02-52d0462ea41d None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:37.333606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b508ad20-fad3-49db-bb02-52d0462ea41d None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:37.333913 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b508ad20-fad3-49db-bb02-52d0462ea41d None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:37.334674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b508ad20-fad3-49db-bb02-52d0462ea41d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:37.334979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b508ad20-fad3-49db-bb02-52d0462ea41d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:37.336356 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b508ad20-fad3-49db-bb02-52d0462ea41d None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:37.348214 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 236/467] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:37.349570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be6107-f884-49d1-a7b9-9db939682cb6 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:37.349857 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be6107-f884-49d1-a7b9-9db939682cb6 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:37.350131 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be6107-f884-49d1-a7b9-9db939682cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:37.365588 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0e0a4-1ff6-416b-93ee-9368b5d9425d None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:37.367171 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-f7be6107-f884-49d1-a7b9-9db939682cb6 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:37.368353 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-f7be6107-f884-49d1-a7b9-9db939682cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:37.393895 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:37.407889 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d817b924-fe6d-4d0b-8403-751eb2bc7c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:37.408811 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817b924-fe6d-4d0b-8403-751eb2bc7c7f None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:37.409291 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817b924-fe6d-4d0b-8403-751eb2bc7c7f None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:37.409723 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817b924-fe6d-4d0b-8403-751eb2bc7c7f None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:37.410614 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d817b924-fe6d-4d0b-8403-751eb2bc7c7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:37.411084 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d817b924-fe6d-4d0b-8403-751eb2bc7c7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:37.412646 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d817b924-fe6d-4d0b-8403-751eb2bc7c7f None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:37.426350 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 232/468] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:37.427373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d817b924-fe6d-4d0b-8403-751eb2bc7c7f None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:37.441112 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 237/469] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:37.441938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be6107-f884-49d1-a7b9-9db939682cb6 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:37.538177 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d817b924-fe6d-4d0b-8403-751eb2bc7c7f None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:37.540435 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:37.555343 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be6107-f884-49d1-a7b9-9db939682cb6 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:37.578055 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5866a27e-2f90-4582-aabe-c7f10b071067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:37.578938 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866a27e-2f90-4582-aabe-c7f10b071067 None admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:37.579398 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866a27e-2f90-4582-aabe-c7f10b071067 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:37.579830 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866a27e-2f90-4582-aabe-c7f10b071067 None admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:37.580708 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5866a27e-2f90-4582-aabe-c7f10b071067 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:37.582326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5866a27e-2f90-4582-aabe-c7f10b071067 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:37.588709 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eeba35-c026-41d6-ac22-7a1c9a44b7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:37.589908 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eeba35-c026-41d6-ac22-7a1c9a44b7d2 None admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:37.590201 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eeba35-c026-41d6-ac22-7a1c9a44b7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:37.590475 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eeba35-c026-41d6-ac22-7a1c9a44b7d2 None admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:37.591182 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eeba35-c026-41d6-ac22-7a1c9a44b7d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:37.592581 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eeba35-c026-41d6-ac22-7a1c9a44b7d2 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:37.603503 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 233/470] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jul 6 13:32:37 2022] POST /identity/v3/users => generated 248 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:32:37.604226 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5866a27e-2f90-4582-aabe-c7f10b071067 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:37.624568 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 238/471] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jul 6 13:32:37 2022] POST /identity/v3/users => generated 246 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:32:37.626051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eeba35-c026-41d6-ac22-7a1c9a44b7d2 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:37.648126 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:37.761696 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eeba35-c026-41d6-ac22-7a1c9a44b7d2 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:37.765628 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5866a27e-2f90-4582-aabe-c7f10b071067 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:37.805053 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da63265-65c4-45f3-9f07-276705ff12b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:37.805846 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da63265-65c4-45f3-9f07-276705ff12b5 None admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:37.806101 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da63265-65c4-45f3-9f07-276705ff12b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:37.806476 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da63265-65c4-45f3-9f07-276705ff12b5 None admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:37.807211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da63265-65c4-45f3-9f07-276705ff12b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:37.809338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da63265-65c4-45f3-9f07-276705ff12b5 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:37.812779 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ee6cb-869e-4857-a0bd-4cd6b6a8556c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:37.813890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ee6cb-869e-4857-a0bd-4cd6b6a8556c None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:37.814386 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ee6cb-869e-4857-a0bd-4cd6b6a8556c None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:37.814844 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ee6cb-869e-4857-a0bd-4cd6b6a8556c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:37.831702 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ee6cb-869e-4857-a0bd-4cd6b6a8556c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:37.833625 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ee6cb-869e-4857-a0bd-4cd6b6a8556c None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:37.841174 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 234/472] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jul 6 13:32:37 2022] POST /identity/v3/users => generated 248 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:32:37.842842 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145ee0d-85a9-41d3-8591-0650763dc862 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:37.843627 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145ee0d-85a9-41d3-8591-0650763dc862 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:37.843627 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145ee0d-85a9-41d3-8591-0650763dc862 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:37.849175 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 235/473] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:37.854715 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5157461-bc52-406e-a4d7-79fae2248c6d None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:37.855750 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5157461-bc52-406e-a4d7-79fae2248c6d None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:37.856040 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5157461-bc52-406e-a4d7-79fae2248c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:37.857284 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-fb4ee6cb-869e-4857-a0bd-4cd6b6a8556c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:37.857284 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:37.857284 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:37.857284 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:37.857284 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:37.857284 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:37.857284 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:37.857284 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:37.857284 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:37.857284 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:37.857284 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:37.857284 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:37.857284 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:37.857284 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:37.857284 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:37.858130 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:37.858130 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:37.858130 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:37.858130 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:37.858130 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:37.858130 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:37.858130 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:37.858130 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:37.858130 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:37.858130 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:37.858130 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:37.858130 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:37.858130 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:37.858130 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:37.858130 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:37.858130 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:37.858130 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:37.859096 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:37.859096 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:37.859096 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:37.859096 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:37.861667 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 239/474] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:37.875527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ee6cb-869e-4857-a0bd-4cd6b6a8556c None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:37.885972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-d5157461-bc52-406e-a4d7-79fae2248c6d None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:37.887390 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-d5157461-bc52-406e-a4d7-79fae2248c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:37.903530 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:37.982134 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:38.003268 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 236/475] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:38.025958 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ee6cb-869e-4857-a0bd-4cd6b6a8556c None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:38.033342 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5157461-bc52-406e-a4d7-79fae2248c6d None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:38.068678 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992529a1-d9fe-4057-89f2-5de7baaeb345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:38.069385 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992529a1-d9fe-4057-89f2-5de7baaeb345 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:38.069665 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992529a1-d9fe-4057-89f2-5de7baaeb345 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:38.069952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992529a1-d9fe-4057-89f2-5de7baaeb345 None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:38.070693 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992529a1-d9fe-4057-89f2-5de7baaeb345 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:38.071005 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992529a1-d9fe-4057-89f2-5de7baaeb345 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:38.072368 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992529a1-d9fe-4057-89f2-5de7baaeb345 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:38.084771 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 240/476] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:38.090243 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992529a1-d9fe-4057-89f2-5de7baaeb345 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:38.174799 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5157461-bc52-406e-a4d7-79fae2248c6d None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:38.205410 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992529a1-d9fe-4057-89f2-5de7baaeb345 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:38.218915 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc73ef-11ce-49da-bee1-3af27babf530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:38.220217 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc73ef-11ce-49da-bee1-3af27babf530 None admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:38.220699 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc73ef-11ce-49da-bee1-3af27babf530 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:38.221368 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc73ef-11ce-49da-bee1-3af27babf530 None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:38.222706 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfc73ef-11ce-49da-bee1-3af27babf530 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:38.224685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfc73ef-11ce-49da-bee1-3af27babf530 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:38.232598 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.sql.core [None req-edfc73ef-11ce-49da-bee1-3af27babf530 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 06 13:32:38.232598 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 06 13:32:38.232598 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [parameters: {'id': '0fdd450bf51d4dc5874bbe0861441809', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 06 13:32:38.232598 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86850) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 06 13:32:38.243836 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:38.257727 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58ad6d-997f-4184-9bc7-83a5f950e424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:38.258448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58ad6d-997f-4184-9bc7-83a5f950e424 None admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:38.258731 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58ad6d-997f-4184-9bc7-83a5f950e424 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:38.259003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58ad6d-997f-4184-9bc7-83a5f950e424 None admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:38.259720 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b58ad6d-997f-4184-9bc7-83a5f950e424 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:38.262414 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b58ad6d-997f-4184-9bc7-83a5f950e424 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:38.266186 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-edfc73ef-11ce-49da-bee1-3af27babf530 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 06 13:32:38.266186 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:38.266186 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 06 13:32:38.266186 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 06 13:32:38.266186 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 06 13:32:38.266186 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 06 13:32:38.266186 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 06 13:32:38.266186 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application result = self._query(query) Jul 06 13:32:38.266186 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 06 13:32:38.266186 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application conn.query(q) Jul 06 13:32:38.266186 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 06 13:32:38.266186 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 06 13:32:38.266186 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 06 13:32:38.266186 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application result.read() Jul 06 13:32:38.266186 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 06 13:32:38.267050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 06 13:32:38.267050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 06 13:32:38.267050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 06 13:32:38.267050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 06 13:32:38.267050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 06 13:32:38.267050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 06 13:32:38.267050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 06 13:32:38.267050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 06 13:32:38.267050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:38.267050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 06 13:32:38.267050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:38.267050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:38.267050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 06 13:32:38.267050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 06 13:32:38.267050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 06 13:32:38.267050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return ref.to_dict() Jul 06 13:32:38.267050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 06 13:32:38.267050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application next(self.gen) Jul 06 13:32:38.267963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 06 13:32:38.267963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application yield resource Jul 06 13:32:38.267963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 06 13:32:38.267963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application next(self.gen) Jul 06 13:32:38.267963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 06 13:32:38.267963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self.session.rollback() Jul 06 13:32:38.267963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 06 13:32:38.267963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self.force_reraise() Jul 06 13:32:38.267963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 06 13:32:38.267963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise self.value Jul 06 13:32:38.267963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 06 13:32:38.267963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 06 13:32:38.267963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 06 13:32:38.267963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application session.commit() Jul 06 13:32:38.267963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 06 13:32:38.267963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 06 13:32:38.267963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 06 13:32:38.268856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._prepare_impl() Jul 06 13:32:38.268856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 06 13:32:38.268856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self.session.flush() Jul 06 13:32:38.268856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 06 13:32:38.268856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._flush(objects) Jul 06 13:32:38.268856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 06 13:32:38.268856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 06 13:32:38.268856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 06 13:32:38.268856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application compat.raise_( Jul 06 13:32:38.268856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 06 13:32:38.268856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception Jul 06 13:32:38.268856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 06 13:32:38.268856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application flush_context.execute() Jul 06 13:32:38.268856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 06 13:32:38.268856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rec.execute(self) Jul 06 13:32:38.268856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 06 13:32:38.268856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 06 13:32:38.269790 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 06 13:32:38.269790 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 06 13:32:38.269790 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 06 13:32:38.269790 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 06 13:32:38.269790 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 06 13:32:38.269790 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 06 13:32:38.269790 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 06 13:32:38.269790 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 06 13:32:38.269790 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 06 13:32:38.269790 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 06 13:32:38.269790 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 06 13:32:38.269790 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 06 13:32:38.269790 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 06 13:32:38.269790 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 06 13:32:38.269790 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 06 13:32:38.269790 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception Jul 06 13:32:38.270624 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 06 13:32:38.270624 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 06 13:32:38.270624 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 06 13:32:38.270624 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 06 13:32:38.270624 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 06 13:32:38.270624 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application result = self._query(query) Jul 06 13:32:38.270624 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 06 13:32:38.270624 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application conn.query(q) Jul 06 13:32:38.270624 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 06 13:32:38.270624 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 06 13:32:38.270624 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 06 13:32:38.270624 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application result.read() Jul 06 13:32:38.270624 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 06 13:32:38.270624 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 06 13:32:38.270624 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 06 13:32:38.270624 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 06 13:32:38.271407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 06 13:32:38.271407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 06 13:32:38.271407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 06 13:32:38.271407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 06 13:32:38.271407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 06 13:32:38.271407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 06 13:32:38.271407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [parameters: {'id': '0fdd450bf51d4dc5874bbe0861441809', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 06 13:32:38.271407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 06 13:32:38.271407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:38.271407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 13:32:38.271407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:38.271407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:38.271407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:38.271407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:38.271407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:38.271407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:38.272193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:38.272193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:38.272193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:38.272193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:38.272193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:38.272193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:38.272193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 06 13:32:38.272193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 06 13:32:38.272193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:38.272193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:38.272193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 06 13:32:38.272193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 06 13:32:38.272193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 06 13:32:38.272193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 06 13:32:38.272193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 06 13:32:38.272193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:38.278522 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 237/477] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Jul 6 13:32:38 2022] POST /identity/v3/roles => generated 144 bytes in 237 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:32:38.280951 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc73ef-11ce-49da-bee1-3af27babf530 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:38.305403 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:38.310411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 241/478] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jul 6 13:32:38 2022] POST /identity/v3/users => generated 249 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:32:38.476764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc73ef-11ce-49da-bee1-3af27babf530 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:38.516361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac089b-61e7-4e6f-a386-69c1fa19e1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:38.517207 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac089b-61e7-4e6f-a386-69c1fa19e1b6 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:38.517494 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac089b-61e7-4e6f-a386-69c1fa19e1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:38.517772 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac089b-61e7-4e6f-a386-69c1fa19e1b6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:38.529466 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac089b-61e7-4e6f-a386-69c1fa19e1b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:38.537563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-b4ac089b-61e7-4e6f-a386-69c1fa19e1b6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 13:32:38.537563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:38.537563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:38.537563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:38.537563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:38.537563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:38.537563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:38.537563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:38.537563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:38.537563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:38.537563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:38.537563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:38.537563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:38.537563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:38.537563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:38.537563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:38.538441 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:38.538441 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:38.538441 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:38.538441 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:38.538441 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:38.538441 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:38.538441 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:38.538441 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:38.538441 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:38.538441 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:38.538441 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:38.538441 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:38.538441 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:38.538441 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:38.538441 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:38.538441 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:38.539236 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:38.539236 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:38.539236 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:38.539236 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:38.539236 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:38.539236 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:38.539236 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:38.539236 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:38.539236 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:38.539236 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:38.539236 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 13:32:38.539236 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:38.539236 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 238/479] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jul 6 13:32:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:38.544618 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58ad6d-997f-4184-9bc7-83a5f950e424 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:38.663460 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58ad6d-997f-4184-9bc7-83a5f950e424 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:38.701746 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94494df8-6792-449a-ba9c-3b8ec4114de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:38.702424 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94494df8-6792-449a-ba9c-3b8ec4114de5 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:38.702712 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94494df8-6792-449a-ba9c-3b8ec4114de5 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:38.703006 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94494df8-6792-449a-ba9c-3b8ec4114de5 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:38.703816 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94494df8-6792-449a-ba9c-3b8ec4114de5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:38.704125 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94494df8-6792-449a-ba9c-3b8ec4114de5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:38.705589 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94494df8-6792-449a-ba9c-3b8ec4114de5 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:38.714262 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 242/480] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:32:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:40.265556 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470a7a0-8895-4f9b-ae7f-b0a71dd88cb4 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:40.266103 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470a7a0-8895-4f9b-ae7f-b0a71dd88cb4 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:40.266103 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470a7a0-8895-4f9b-ae7f-b0a71dd88cb4 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:40.267681 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 239/481] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:40.273051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4b799-e3e4-4cef-ac9e-0ddcc0555c9d None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:40.273457 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4b799-e3e4-4cef-ac9e-0ddcc0555c9d None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:40.273663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4b799-e3e4-4cef-ac9e-0ddcc0555c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:40.301864 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-d2f4b799-e3e4-4cef-ac9e-0ddcc0555c9d None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:40.303287 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-d2f4b799-e3e4-4cef-ac9e-0ddcc0555c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:40.382564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:40.414826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 243/482] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 13:32:40.437202 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb758b5-80f5-4b67-bba1-688003e85c2f None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:40.437484 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb758b5-80f5-4b67-bba1-688003e85c2f None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:40.437761 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb758b5-80f5-4b67-bba1-688003e85c2f None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:40.439315 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 240/483] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:40.444533 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6b459-acc4-4239-a52d-be75116e235e None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:40.444648 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445d323-d92c-4214-92d5-67094f84c0fe None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:40.444948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445d323-d92c-4214-92d5-67094f84c0fe None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:40.445280 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445d323-d92c-4214-92d5-67094f84c0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:40.445591 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6b459-acc4-4239-a52d-be75116e235e None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:40.446080 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6b459-acc4-4239-a52d-be75116e235e None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:40.447780 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 241/484] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:40.453682 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac9a74-cb23-48d0-b0c4-12eb4273f1ec None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:40.454223 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac9a74-cb23-48d0-b0c4-12eb4273f1ec None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:40.454723 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac9a74-cb23-48d0-b0c4-12eb4273f1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:40.465286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-1445d323-d92c-4214-92d5-67094f84c0fe None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:40.466912 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-1445d323-d92c-4214-92d5-67094f84c0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:40.475929 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-1aac9a74-cb23-48d0-b0c4-12eb4273f1ec None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:40.477416 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-1aac9a74-cb23-48d0-b0c4-12eb4273f1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:40.535840 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 244/485] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 13:32:40.536805 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445d323-d92c-4214-92d5-67094f84c0fe None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:40.546944 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:40.553584 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 242/486] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 13:32:40.564650 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac9a74-cb23-48d0-b0c4-12eb4273f1ec None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:40.667192 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445d323-d92c-4214-92d5-67094f84c0fe None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:40.677314 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:40.694135 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac9a74-cb23-48d0-b0c4-12eb4273f1ec None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:40.705639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788fd37-1702-406a-9110-754ef87a2414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:40.706689 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788fd37-1702-406a-9110-754ef87a2414 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:40.707124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788fd37-1702-406a-9110-754ef87a2414 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:40.707686 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788fd37-1702-406a-9110-754ef87a2414 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:40.719005 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788fd37-1702-406a-9110-754ef87a2414 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:40.727074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-1788fd37-1702-406a-9110-754ef87a2414 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:32:40.727074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:40.727074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:40.727074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:40.727074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:40.727074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:40.727074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:40.727074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:40.727074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:40.727074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:40.727074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:40.727074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:40.727074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:40.727074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:40.727074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:40.727074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:40.728094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:40.728094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:40.728094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:40.728094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:40.728094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:40.728094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:40.728094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:40.728094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:40.728094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:40.728094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:40.728094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:40.728094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:40.728094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:40.728094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:40.728094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:40.728094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:40.728912 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:40.728912 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:40.728912 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:40.728912 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:40.728912 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:40.728912 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:40.728912 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:40.728912 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:40.728912 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:40.728912 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:40.728912 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:32:40.728912 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:40.729955 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 245/487] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jul 6 13:32:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:40.730927 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788fd37-1702-406a-9110-754ef87a2414 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:40.737101 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c97fea9-aace-42bb-8985-2b410bf781e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:40.737915 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97fea9-aace-42bb-8985-2b410bf781e9 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:40.738194 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97fea9-aace-42bb-8985-2b410bf781e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:40.738465 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97fea9-aace-42bb-8985-2b410bf781e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:40.750389 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c97fea9-aace-42bb-8985-2b410bf781e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:40.758470 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-2c97fea9-aace-42bb-8985-2b410bf781e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:32:40.758470 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:40.758470 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:40.758470 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:40.758470 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:40.758470 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:40.758470 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:40.758470 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:40.758470 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:40.758470 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:40.758470 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:40.758470 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:40.758470 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:40.758470 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:40.758470 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:40.758470 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:40.759387 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:40.759387 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:40.759387 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:40.759387 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:40.759387 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:40.759387 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:40.759387 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:40.759387 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:40.759387 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:40.759387 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:40.759387 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:40.759387 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:40.759387 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:40.759387 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:40.759387 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:40.759387 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:40.760201 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:40.760201 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:40.760201 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:40.760201 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:40.760201 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:40.760201 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:40.760201 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:40.760201 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:40.760201 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:40.760201 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:40.760201 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:32:40.760201 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:40.760201 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 243/488] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jul 6 13:32:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:32:40.761921 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c97fea9-aace-42bb-8985-2b410bf781e9 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:40.834625 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:40.847489 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788fd37-1702-406a-9110-754ef87a2414 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:40.877770 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c97fea9-aace-42bb-8985-2b410bf781e9 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:40.884187 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ecb699-f17c-4883-9934-e3ec8a15b0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:40.885268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecb699-f17c-4883-9934-e3ec8a15b0f2 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:40.885786 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecb699-f17c-4883-9934-e3ec8a15b0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:40.886345 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecb699-f17c-4883-9934-e3ec8a15b0f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:40.897853 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ecb699-f17c-4883-9934-e3ec8a15b0f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:40.906323 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-40ecb699-f17c-4883-9934-e3ec8a15b0f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:32:40.906323 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:40.906323 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:40.906323 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:40.906323 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:40.906323 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:40.906323 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:40.906323 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:40.906323 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:40.906323 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:40.906323 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:40.906323 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:40.906323 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:40.906323 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:40.906323 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:40.906323 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:40.907232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:40.907232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:40.907232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:40.907232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:40.907232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:40.907232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:40.907232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:40.907232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:40.907232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:40.907232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:40.907232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:40.907232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:40.907232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:40.907232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:40.907232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:40.907232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:40.908041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:40.908041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:40.908041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:40.908041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:40.908041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:40.908041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:40.908041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:40.908041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:40.908041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:40.908041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:40.908041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:32:40.908041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:40.910702 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 246/489] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jul 6 13:32:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:40.911430 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ecb699-f17c-4883-9934-e3ec8a15b0f2 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:40.921642 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f056c18d-f135-4275-a858-f67674b22785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:40.922321 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056c18d-f135-4275-a858-f67674b22785 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:40.922655 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056c18d-f135-4275-a858-f67674b22785 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:40.922934 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056c18d-f135-4275-a858-f67674b22785 None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:40.923693 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f056c18d-f135-4275-a858-f67674b22785 None admin] RBAC: Adding query filter params (name=service) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:40.924255 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f056c18d-f135-4275-a858-f67674b22785 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:40.925709 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f056c18d-f135-4275-a858-f67674b22785 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:40.935354 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 244/490] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jul 6 13:32:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:40.936300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f056c18d-f135-4275-a858-f67674b22785 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:40.952445 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:41.035084 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ecb699-f17c-4883-9934-e3ec8a15b0f2 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:41.045784 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f056c18d-f135-4275-a858-f67674b22785 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:41.069786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:41.072690 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172df52-ce2b-4bf2-969f-d91b4755afc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:41.074245 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172df52-ce2b-4bf2-969f-d91b4755afc7 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:41.074748 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172df52-ce2b-4bf2-969f-d91b4755afc7 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:41.075236 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172df52-ce2b-4bf2-969f-d91b4755afc7 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:41.077203 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172df52-ce2b-4bf2-969f-d91b4755afc7 None admin] RBAC: Adding query filter params (name=service) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:41.077663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172df52-ce2b-4bf2-969f-d91b4755afc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:41.079511 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172df52-ce2b-4bf2-969f-d91b4755afc7 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:41.089383 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee2a5b-f40c-4412-a2ee-81c816ad9e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:41.090014 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee2a5b-f40c-4412-a2ee-81c816ad9e2b None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:41.090418 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee2a5b-f40c-4412-a2ee-81c816ad9e2b None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:41.092269 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee2a5b-f40c-4412-a2ee-81c816ad9e2b None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:41.093837 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee2a5b-f40c-4412-a2ee-81c816ad9e2b None admin] RBAC: Adding query filter params (name=service) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:41.094159 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee2a5b-f40c-4412-a2ee-81c816ad9e2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:41.095567 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee2a5b-f40c-4412-a2ee-81c816ad9e2b None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:41.104081 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 247/491] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jul 6 13:32:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:41.104081 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172df52-ce2b-4bf2-969f-d91b4755afc7 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:41.110587 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 245/492] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jul 6 13:32:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:41.111792 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee2a5b-f40c-4412-a2ee-81c816ad9e2b None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:41.220436 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee2a5b-f40c-4412-a2ee-81c816ad9e2b None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:41.222777 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:41.234973 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172df52-ce2b-4bf2-969f-d91b4755afc7 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:41.256221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bab066-3bff-441c-a787-e44ea8387680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:41.257047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bab066-3bff-441c-a787-e44ea8387680 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:41.257331 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bab066-3bff-441c-a787-e44ea8387680 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:41.257599 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bab066-3bff-441c-a787-e44ea8387680 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:41.268585 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bab066-3bff-441c-a787-e44ea8387680 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:41.270008 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bab066-3bff-441c-a787-e44ea8387680 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:41.280830 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b05d9-17c9-48d5-b3a5-23b60ad870cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:41.281692 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b05d9-17c9-48d5-b3a5-23b60ad870cb None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:41.281979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b05d9-17c9-48d5-b3a5-23b60ad870cb None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:41.282313 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b05d9-17c9-48d5-b3a5-23b60ad870cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:41.288068 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-c5bab066-3bff-441c-a787-e44ea8387680 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:41.288068 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:41.288068 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:41.288068 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:41.288068 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:41.288068 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:41.288068 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:41.288068 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:41.288068 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:41.288068 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:41.288068 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:41.288068 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:41.288068 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:41.288068 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:41.288068 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:41.288876 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:41.288876 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:41.288876 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:41.288876 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:41.288876 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:41.288876 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:41.288876 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:41.288876 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:41.288876 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:41.288876 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:41.288876 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:41.288876 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:41.288876 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:41.288876 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:41.288876 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:41.288876 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:41.288876 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:41.289764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:41.289764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:41.289764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:41.289764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:41.294867 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 246/493] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:41.295865 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bab066-3bff-441c-a787-e44ea8387680 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:41.296661 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b05d9-17c9-48d5-b3a5-23b60ad870cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:41.300322 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b05d9-17c9-48d5-b3a5-23b60ad870cb None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:41.318564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-202b05d9-17c9-48d5-b3a5-23b60ad870cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:41.318564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:41.318564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:41.318564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:41.318564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:41.318564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:41.318564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:41.318564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:41.318564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:41.318564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:41.318564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:41.318564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:41.318564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:41.318564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:41.318564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:41.319632 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:41.319632 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:41.319632 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:41.319632 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:41.319632 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:41.319632 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:41.319632 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:41.319632 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:41.319632 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:41.319632 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:41.319632 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:41.319632 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:41.319632 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:41.319632 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:41.319632 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:41.319632 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:41.319632 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:41.320503 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:41.320503 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:41.320503 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:41.320503 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:41.320503 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 248/494] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:41.321285 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d646e54-7778-42d7-a9e2-a7fd5019c6d8 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:41.321571 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d646e54-7778-42d7-a9e2-a7fd5019c6d8 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:41.321847 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d646e54-7778-42d7-a9e2-a7fd5019c6d8 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:41.323224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 249/495] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:41.325765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d646e54-7778-42d7-a9e2-a7fd5019c6d8 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:41.337094 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:41.402797 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bab066-3bff-441c-a787-e44ea8387680 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:41.428305 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d646e54-7778-42d7-a9e2-a7fd5019c6d8 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:41.436781 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee1b7b8-8804-41a1-881a-1ed6069b50dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:41.437605 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1b7b8-8804-41a1-881a-1ed6069b50dc None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:41.437894 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1b7b8-8804-41a1-881a-1ed6069b50dc None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:41.438170 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1b7b8-8804-41a1-881a-1ed6069b50dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:41.448499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee1b7b8-8804-41a1-881a-1ed6069b50dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:41.450454 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee1b7b8-8804-41a1-881a-1ed6069b50dc None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:41.462774 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2792f-0ab0-4a7d-b79e-65b5857a90de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:41.463440 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2792f-0ab0-4a7d-b79e-65b5857a90de None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:41.463730 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2792f-0ab0-4a7d-b79e-65b5857a90de None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:41.464009 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2792f-0ab0-4a7d-b79e-65b5857a90de None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:41.465303 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2792f-0ab0-4a7d-b79e-65b5857a90de None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:41.465618 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2792f-0ab0-4a7d-b79e-65b5857a90de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:41.467241 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2792f-0ab0-4a7d-b79e-65b5857a90de None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:41.469162 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-2ee1b7b8-8804-41a1-881a-1ed6069b50dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:41.469162 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:41.469162 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:41.469162 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:41.469162 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:41.469162 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:41.469162 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:41.469162 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:41.469162 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:41.469162 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:41.469162 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:41.469162 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:41.469162 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:41.469162 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:41.469162 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:41.470006 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:41.470006 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:41.470006 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:41.470006 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:41.470006 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:41.470006 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:41.470006 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:41.470006 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:41.470006 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:41.470006 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:41.470006 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:41.470006 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:41.470006 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:41.470006 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:41.470006 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:41.470006 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:41.470006 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:41.471055 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:41.471055 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:41.471055 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:41.471055 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:41.471055 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 247/496] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:41.471606 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397afe0-c2c9-436a-bfd9-85cd1f9a7b7c None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:41.471880 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397afe0-c2c9-436a-bfd9-85cd1f9a7b7c None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:41.472146 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397afe0-c2c9-436a-bfd9-85cd1f9a7b7c None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:41.473429 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 248/497] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:41.474123 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6397afe0-c2c9-436a-bfd9-85cd1f9a7b7c None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:41.487228 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 250/498] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:41.488589 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc86ecb-e606-4f5d-8c61-ec084ca3d8f6 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:41.488877 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc86ecb-e606-4f5d-8c61-ec084ca3d8f6 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:41.489196 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc86ecb-e606-4f5d-8c61-ec084ca3d8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:41.494019 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:41.506249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-5cc86ecb-e606-4f5d-8c61-ec084ca3d8f6 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:41.507425 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-5cc86ecb-e606-4f5d-8c61-ec084ca3d8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:41.565918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:41.572665 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 251/499] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:41.573526 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc86ecb-e606-4f5d-8c61-ec084ca3d8f6 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:41.596773 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6397afe0-c2c9-436a-bfd9-85cd1f9a7b7c None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:41.629844 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0b99b-3f78-4dc5-ac15-9cfc54f90f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:41.630499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0b99b-3f78-4dc5-ac15-9cfc54f90f0c None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:41.630782 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0b99b-3f78-4dc5-ac15-9cfc54f90f0c None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:41.631055 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0b99b-3f78-4dc5-ac15-9cfc54f90f0c None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:41.631770 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce0b99b-3f78-4dc5-ac15-9cfc54f90f0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:41.632074 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce0b99b-3f78-4dc5-ac15-9cfc54f90f0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:41.633476 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce0b99b-3f78-4dc5-ac15-9cfc54f90f0c None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:41.644372 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 249/500] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:41.645381 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fcf571-82ed-4287-9dbf-8b439e691783 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:41.645667 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fcf571-82ed-4287-9dbf-8b439e691783 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:41.645943 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fcf571-82ed-4287-9dbf-8b439e691783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:41.662275 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-09fcf571-82ed-4287-9dbf-8b439e691783 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:41.663370 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-09fcf571-82ed-4287-9dbf-8b439e691783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:41.675638 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc86ecb-e606-4f5d-8c61-ec084ca3d8f6 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:41.704376 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:41.710702 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ffee05-057a-4262-b943-8e4ffd2626a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:41.711358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ffee05-057a-4262-b943-8e4ffd2626a3 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:41.711637 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ffee05-057a-4262-b943-8e4ffd2626a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:41.711909 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ffee05-057a-4262-b943-8e4ffd2626a3 None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:41.712633 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ffee05-057a-4262-b943-8e4ffd2626a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:41.713191 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ffee05-057a-4262-b943-8e4ffd2626a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:41.715170 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ffee05-057a-4262-b943-8e4ffd2626a3 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:41.726006 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 252/501] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:41.726773 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ffee05-057a-4262-b943-8e4ffd2626a3 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:41.733770 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 250/502] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:41.733770 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fcf571-82ed-4287-9dbf-8b439e691783 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:41.809644 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:41.832663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ffee05-057a-4262-b943-8e4ffd2626a3 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:41.833624 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fcf571-82ed-4287-9dbf-8b439e691783 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:41.866541 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae1e9f-b898-4aff-aa04-eb4f4766c26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:41.867323 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae1e9f-b898-4aff-aa04-eb4f4766c26e None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:41.867603 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae1e9f-b898-4aff-aa04-eb4f4766c26e None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:41.867698 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742ea84-fa87-4507-a8ff-109b47c681c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:41.867877 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae1e9f-b898-4aff-aa04-eb4f4766c26e None admin] PATH_INFO: `/v3/users/glance` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:41.868395 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742ea84-fa87-4507-a8ff-109b47c681c6 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:41.868680 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742ea84-fa87-4507-a8ff-109b47c681c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:41.868956 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742ea84-fa87-4507-a8ff-109b47c681c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:41.875661 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae1e9f-b898-4aff-aa04-eb4f4766c26e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:41.877128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae1e9f-b898-4aff-aa04-eb4f4766c26e None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:41.879361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7742ea84-fa87-4507-a8ff-109b47c681c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:41.887540 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-7742ea84-fa87-4507-a8ff-109b47c681c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:32:41.887540 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:41.887540 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:41.887540 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:41.887540 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:41.887540 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:41.887540 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:41.887540 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:41.887540 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:41.887540 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:41.887540 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:41.887540 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:41.887540 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:41.887540 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:41.887540 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:41.887540 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:41.888563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:41.888563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:41.888563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:41.888563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:41.888563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:41.888563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:41.888563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:41.888563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:41.888563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:41.888563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:41.888563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:41.888563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:41.888563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:41.888563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:41.888563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:41.888563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:41.889478 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:41.889478 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:41.889478 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:41.889478 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:41.889478 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:41.889478 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:41.889478 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:41.889478 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:41.889478 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:41.889478 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:41.889478 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:32:41.889478 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:41.889478 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 251/503] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jul 6 13:32:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:41.890118 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742ea84-fa87-4507-a8ff-109b47c681c6 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:41.895083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-4dae1e9f-b898-4aff-aa04-eb4f4766c26e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 06 13:32:41.895083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:41.895083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:41.895083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:41.895083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:41.895083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:41.895083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:41.895083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:41.895083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:41.895083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:41.895083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:41.895083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:41.895083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:32:41.895083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:32:41.895083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:32:41.895083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:32:41.896105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:41.896105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:41.896105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:32:41.896105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:32:41.896105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:32:41.896105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:32:41.896105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:41.896105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:41.896105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:41.896105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:41.896105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:41.896105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:41.896105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:41.896105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:41.896105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:41.896105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:41.896105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:41.896105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:41.896988 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:41.896988 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:41.896988 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:32:41.896988 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:32:41.896988 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:32:41.896988 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:32:41.896988 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:32:41.896988 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:32:41.896988 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 06 13:32:41.896988 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:41.896988 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 253/504] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:32:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:41.897591 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae1e9f-b898-4aff-aa04-eb4f4766c26e None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:41.949511 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:41.995343 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae1e9f-b898-4aff-aa04-eb4f4766c26e None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:41.996395 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742ea84-fa87-4507-a8ff-109b47c681c6 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:42.029230 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c58be-ddba-4fb2-bb1a-d7969fea4fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:42.029926 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4d60e-a620-40ed-8f75-92df31edac90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:42.030081 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c58be-ddba-4fb2-bb1a-d7969fea4fb8 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:42.030268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c58be-ddba-4fb2-bb1a-d7969fea4fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:42.030539 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c58be-ddba-4fb2-bb1a-d7969fea4fb8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:42.030715 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4d60e-a620-40ed-8f75-92df31edac90 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:42.031000 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4d60e-a620-40ed-8f75-92df31edac90 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:42.031278 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4d60e-a620-40ed-8f75-92df31edac90 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:42.037810 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c58be-ddba-4fb2-bb1a-d7969fea4fb8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:42.038996 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe4d60e-a620-40ed-8f75-92df31edac90 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:42.039275 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c58be-ddba-4fb2-bb1a-d7969fea4fb8 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:42.040420 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe4d60e-a620-40ed-8f75-92df31edac90 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:42.042125 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:42.053720 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-cc9c58be-ddba-4fb2-bb1a-d7969fea4fb8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 06 13:32:42.053720 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:42.053720 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:42.053720 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:42.053720 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:42.053720 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:42.053720 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:42.053720 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:42.053720 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:42.053720 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:42.053720 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:42.053720 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:42.053720 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:32:42.053720 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:32:42.053720 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:32:42.053720 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:32:42.054629 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:42.054629 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:42.054629 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:32:42.054629 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:32:42.054629 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:32:42.054629 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:32:42.054629 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:42.054629 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:42.054629 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:42.054629 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:42.054629 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:42.054629 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:42.054629 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:42.054629 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:42.054629 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:42.054629 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:42.054629 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:42.054629 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:42.055505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:42.055505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:42.055505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:32:42.055505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:32:42.055505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:32:42.055505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:32:42.055505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:32:42.055505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:32:42.055505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 06 13:32:42.055505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:42.055505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 254/505] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Jul 6 13:32:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:42.056067 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c58be-ddba-4fb2-bb1a-d7969fea4fb8 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:42.060912 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-bbe4d60e-a620-40ed-8f75-92df31edac90 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 06 13:32:42.060912 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:42.060912 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:42.060912 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:42.060912 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:42.060912 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:42.060912 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:42.060912 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:42.060912 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:42.060912 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:42.060912 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:42.060912 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:42.060912 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:32:42.060912 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:32:42.060912 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:32:42.060912 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:32:42.061967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:42.061967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:42.061967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:32:42.061967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:32:42.061967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:32:42.061967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:32:42.061967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:42.061967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:42.061967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:42.061967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:42.061967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:42.061967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:42.061967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:42.061967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:42.061967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:42.061967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:42.061967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:42.061967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:42.062862 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:42.062862 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:42.062862 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:32:42.062862 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:32:42.062862 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:32:42.062862 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:32:42.062862 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:32:42.062862 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:32:42.062862 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 06 13:32:42.062862 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:42.062862 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 252/506] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:32:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:42.063419 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4d60e-a620-40ed-8f75-92df31edac90 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:42.165587 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4d60e-a620-40ed-8f75-92df31edac90 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:42.165869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c58be-ddba-4fb2-bb1a-d7969fea4fb8 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:42.178469 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:42.199879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05060f77-cea1-4fd2-b4e3-e4655f8cfb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:42.200039 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1f751-d593-4d69-99e3-63b75f862fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:42.200479 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05060f77-cea1-4fd2-b4e3-e4655f8cfb29 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:42.200709 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1f751-d593-4d69-99e3-63b75f862fd8 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:42.200797 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05060f77-cea1-4fd2-b4e3-e4655f8cfb29 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:42.200991 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1f751-d593-4d69-99e3-63b75f862fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:42.201112 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05060f77-cea1-4fd2-b4e3-e4655f8cfb29 None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:42.201295 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1f751-d593-4d69-99e3-63b75f862fd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:42.201787 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05060f77-cea1-4fd2-b4e3-e4655f8cfb29 None admin] RBAC: Adding query filter params (name=service) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:42.202098 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05060f77-cea1-4fd2-b4e3-e4655f8cfb29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:42.203481 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05060f77-cea1-4fd2-b4e3-e4655f8cfb29 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:42.211267 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 253/507] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jul 6 13:32:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:42.211570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1f751-d593-4d69-99e3-63b75f862fd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:42.212089 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05060f77-cea1-4fd2-b4e3-e4655f8cfb29 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:42.213009 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1f751-d593-4d69-99e3-63b75f862fd8 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:42.229914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-b5f1f751-d593-4d69-99e3-63b75f862fd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:42.229914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:42.229914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:42.229914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:42.229914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:42.229914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:42.229914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:42.229914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:42.229914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:42.229914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:42.229914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:42.229914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:42.229914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:42.229914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:42.229914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:42.230974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:42.230974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:42.230974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:42.230974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:42.230974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:42.230974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:42.230974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:42.230974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:42.230974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:42.230974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:42.230974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:42.230974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:42.230974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:42.230974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:42.230974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:42.230974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:42.230974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:42.231806 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:42.231806 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:42.231806 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:42.231806 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:42.231806 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 255/508] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:42.232074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1f751-d593-4d69-99e3-63b75f862fd8 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:42.274338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:42.317904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05060f77-cea1-4fd2-b4e3-e4655f8cfb29 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:42.334569 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1f751-d593-4d69-99e3-63b75f862fd8 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:42.350648 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d0e9a-4e19-4ff0-9da3-3d05b0b08287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:42.351315 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d0e9a-4e19-4ff0-9da3-3d05b0b08287 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:42.351594 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d0e9a-4e19-4ff0-9da3-3d05b0b08287 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:42.351867 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d0e9a-4e19-4ff0-9da3-3d05b0b08287 None admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:42.352693 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d0e9a-4e19-4ff0-9da3-3d05b0b08287 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:42.353012 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d0e9a-4e19-4ff0-9da3-3d05b0b08287 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:42.354436 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d0e9a-4e19-4ff0-9da3-3d05b0b08287 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:42.367491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f73a24-ed54-4f90-b8af-957801fbb55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:42.368160 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f73a24-ed54-4f90-b8af-957801fbb55a None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:42.368445 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f73a24-ed54-4f90-b8af-957801fbb55a None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:42.368725 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f73a24-ed54-4f90-b8af-957801fbb55a None admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:42.369607 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f73a24-ed54-4f90-b8af-957801fbb55a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:42.369923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f73a24-ed54-4f90-b8af-957801fbb55a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:42.371947 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f73a24-ed54-4f90-b8af-957801fbb55a None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:42.390868 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 254/509] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Jul 6 13:32:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:42.391688 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d0e9a-4e19-4ff0-9da3-3d05b0b08287 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:42.414561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 256/510] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Jul 6 13:32:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:42.415399 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f73a24-ed54-4f90-b8af-957801fbb55a None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:42.440734 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:42.490561 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d0e9a-4e19-4ff0-9da3-3d05b0b08287 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:42.508779 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:42.518424 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f73a24-ed54-4f90-b8af-957801fbb55a None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:42.525640 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00a43e-2edf-47c3-a892-1256c5081e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:42.526306 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00a43e-2edf-47c3-a892-1256c5081e0f None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:42.526586 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00a43e-2edf-47c3-a892-1256c5081e0f None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:42.526923 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00a43e-2edf-47c3-a892-1256c5081e0f None admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:42.528012 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00a43e-2edf-47c3-a892-1256c5081e0f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:42.528323 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00a43e-2edf-47c3-a892-1256c5081e0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:42.529785 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00a43e-2edf-47c3-a892-1256c5081e0f None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:42.546666 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 255/511] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Jul 6 13:32:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:42.547448 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00a43e-2edf-47c3-a892-1256c5081e0f None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:42.552021 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfaacde-7b25-4897-8c20-77ce49ae4b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:42.552806 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfaacde-7b25-4897-8c20-77ce49ae4b3a None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:42.553126 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfaacde-7b25-4897-8c20-77ce49ae4b3a None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:42.553402 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfaacde-7b25-4897-8c20-77ce49ae4b3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:42.563825 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfaacde-7b25-4897-8c20-77ce49ae4b3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:42.565299 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfaacde-7b25-4897-8c20-77ce49ae4b3a None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:42.582405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-2bfaacde-7b25-4897-8c20-77ce49ae4b3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:42.582405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:42.582405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:42.582405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:42.582405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:42.582405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:42.582405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:42.582405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:42.582405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:42.582405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:42.582405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:42.582405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:42.582405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:42.582405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:42.582405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:42.583441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:42.583441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:42.583441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:42.583441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:42.583441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:42.583441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:42.583441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:42.583441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:42.583441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:42.583441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:42.583441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:42.583441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:42.583441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:42.583441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:42.583441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:42.583441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:42.583441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:42.584290 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:42.584290 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:42.584290 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:42.584290 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:42.584290 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 257/512] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:42.584290 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfaacde-7b25-4897-8c20-77ce49ae4b3a None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:42.648637 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00a43e-2edf-47c3-a892-1256c5081e0f None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:42.659180 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:42.682977 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebc493-acd7-4353-920b-b3d281c94378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:42.683703 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebc493-acd7-4353-920b-b3d281c94378 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:42.683986 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebc493-acd7-4353-920b-b3d281c94378 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:42.684259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebc493-acd7-4353-920b-b3d281c94378 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:42.684410 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfaacde-7b25-4897-8c20-77ce49ae4b3a None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:42.684976 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebc493-acd7-4353-920b-b3d281c94378 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:42.685310 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebc493-acd7-4353-920b-b3d281c94378 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:42.686688 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebc493-acd7-4353-920b-b3d281c94378 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:42.698000 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 256/513] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:42.699127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebc493-acd7-4353-920b-b3d281c94378 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:42.722197 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99331d1f-3566-4b7c-ae38-a039843b1b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:42.722844 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99331d1f-3566-4b7c-ae38-a039843b1b87 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:42.723129 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99331d1f-3566-4b7c-ae38-a039843b1b87 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:42.723407 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99331d1f-3566-4b7c-ae38-a039843b1b87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:42.733067 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:42.789869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99331d1f-3566-4b7c-ae38-a039843b1b87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:42.791308 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99331d1f-3566-4b7c-ae38-a039843b1b87 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:42.796399 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebc493-acd7-4353-920b-b3d281c94378 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:42.829532 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcea1ee-cd10-46e7-8ae6-b7630ef44e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:42.830180 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcea1ee-cd10-46e7-8ae6-b7630ef44e67 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:42.830461 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcea1ee-cd10-46e7-8ae6-b7630ef44e67 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:42.830736 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcea1ee-cd10-46e7-8ae6-b7630ef44e67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:42.863912 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:42.891256 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 258/514] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:32:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:32:42.892080 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99331d1f-3566-4b7c-ae38-a039843b1b87 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:42.896557 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcea1ee-cd10-46e7-8ae6-b7630ef44e67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:42.898039 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcea1ee-cd10-46e7-8ae6-b7630ef44e67 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:42.931184 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:42.989704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99331d1f-3566-4b7c-ae38-a039843b1b87 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:42.996478 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 257/515] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:32:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:32:42.997482 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcea1ee-cd10-46e7-8ae6-b7630ef44e67 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:43.024242 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e27c5-1076-43e4-995b-d5b8fcde4371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:43.024884 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e27c5-1076-43e4-995b-d5b8fcde4371 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:43.025203 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e27c5-1076-43e4-995b-d5b8fcde4371 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:43.025487 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e27c5-1076-43e4-995b-d5b8fcde4371 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:43.067597 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:43.093332 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5e27c5-1076-43e4-995b-d5b8fcde4371 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:43.094800 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5e27c5-1076-43e4-995b-d5b8fcde4371 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:43.098503 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcea1ee-cd10-46e7-8ae6-b7630ef44e67 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:43.131140 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b1ee6-c8cd-487a-a7d3-6e0a0c9041e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:43.131789 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b1ee6-c8cd-487a-a7d3-6e0a0c9041e7 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:43.132070 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b1ee6-c8cd-487a-a7d3-6e0a0c9041e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:43.132344 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b1ee6-c8cd-487a-a7d3-6e0a0c9041e7 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:43.133084 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b1ee6-c8cd-487a-a7d3-6e0a0c9041e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:43.133397 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b1ee6-c8cd-487a-a7d3-6e0a0c9041e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:43.134780 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b1ee6-c8cd-487a-a7d3-6e0a0c9041e7 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:43.137064 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:43.146608 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 258/516] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:43.147557 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b1ee6-c8cd-487a-a7d3-6e0a0c9041e7 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:43.196562 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 259/517] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:32:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:32:43.197437 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e27c5-1076-43e4-995b-d5b8fcde4371 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:43.255237 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b1ee6-c8cd-487a-a7d3-6e0a0c9041e7 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:43.269283 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:43.288878 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5d5d5-f19b-4634-a594-e327cb240c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:43.289596 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5d5d5-f19b-4634-a594-e327cb240c5f None admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:43.289876 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5d5d5-f19b-4634-a594-e327cb240c5f None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:43.290149 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5d5d5-f19b-4634-a594-e327cb240c5f None admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:43.290856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a5d5d5-f19b-4634-a594-e327cb240c5f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:43.292879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a5d5d5-f19b-4634-a594-e327cb240c5f None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:43.298246 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e27c5-1076-43e4-995b-d5b8fcde4371 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:43.313432 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 259/518] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jul 6 13:32:43 2022] POST /identity/v3/users => generated 247 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:32:43.314280 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5d5d5-f19b-4634-a594-e327cb240c5f None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:43.352024 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd8665-0907-49cf-8e33-cd1c4f94fc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:43.352738 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd8665-0907-49cf-8e33-cd1c4f94fc99 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:43.353039 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd8665-0907-49cf-8e33-cd1c4f94fc99 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:43.353318 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd8665-0907-49cf-8e33-cd1c4f94fc99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:43.365042 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cd8665-0907-49cf-8e33-cd1c4f94fc99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:43.366532 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cd8665-0907-49cf-8e33-cd1c4f94fc99 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:43.375931 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:43.387767 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-07cd8665-0907-49cf-8e33-cd1c4f94fc99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:43.387767 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:43.387767 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:43.387767 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:43.387767 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:43.387767 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:43.387767 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:43.387767 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:43.387767 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:43.387767 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:43.387767 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:43.387767 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:43.387767 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:43.387767 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:43.387767 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:43.388730 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:43.388730 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:43.388730 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:43.388730 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:43.388730 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:43.388730 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:43.388730 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:43.388730 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:43.388730 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:43.388730 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:43.388730 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:43.388730 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:43.388730 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:43.388730 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:43.388730 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:43.388730 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:43.388730 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:43.389621 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:43.389621 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:43.389621 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:43.389621 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:43.389621 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 260/519] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:43.389898 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd8665-0907-49cf-8e33-cd1c4f94fc99 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:43.437851 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5d5d5-f19b-4634-a594-e327cb240c5f None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:43.479861 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ac25e-8898-4331-8b24-3553ca84ce0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:43.480671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ac25e-8898-4331-8b24-3553ca84ce0f None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:43.480953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ac25e-8898-4331-8b24-3553ca84ce0f None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:43.481259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ac25e-8898-4331-8b24-3553ca84ce0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:43.493790 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30ac25e-8898-4331-8b24-3553ca84ce0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:43.495250 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30ac25e-8898-4331-8b24-3553ca84ce0f None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:43.506948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd8665-0907-49cf-8e33-cd1c4f94fc99 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:43.517571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-c30ac25e-8898-4331-8b24-3553ca84ce0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:43.517571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:43.517571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:43.517571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:43.517571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:43.517571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:43.517571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:43.517571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:43.517571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:43.517571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:43.517571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:43.517571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:43.517571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:43.517571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:43.517571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:43.518462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:43.518462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:43.518462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:43.518462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:43.518462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:43.518462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:43.518462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:43.518462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:43.518462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:43.518462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:43.518462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:43.518462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:43.518462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:43.518462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:43.518462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:43.518462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:43.518462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:43.519305 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:43.519305 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:43.519305 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:43.519305 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:43.519305 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 260/520] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:43.521672 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ac25e-8898-4331-8b24-3553ca84ce0f None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:43.538732 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:43.549280 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec409ee-60a7-4f54-a9d3-c5c13e51616d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:43.550082 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec409ee-60a7-4f54-a9d3-c5c13e51616d None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:43.550371 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec409ee-60a7-4f54-a9d3-c5c13e51616d None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:43.550767 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec409ee-60a7-4f54-a9d3-c5c13e51616d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:43.558997 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec409ee-60a7-4f54-a9d3-c5c13e51616d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:43.560542 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec409ee-60a7-4f54-a9d3-c5c13e51616d None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:43.576675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-0ec409ee-60a7-4f54-a9d3-c5c13e51616d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 06 13:32:43.576675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:43.576675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:43.576675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:43.576675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:43.576675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:43.576675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:43.576675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:43.576675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:43.576675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:43.576675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:43.576675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:43.576675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:32:43.576675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:32:43.576675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:32:43.576675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:32:43.577663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:43.577663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:43.577663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:32:43.577663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:32:43.577663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:32:43.577663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:32:43.577663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:43.577663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:43.577663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:43.577663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:43.577663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:43.577663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:43.577663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:43.577663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:43.577663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:43.577663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:43.577663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:43.577663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:43.578893 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:43.578893 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:43.578893 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:32:43.578893 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:32:43.578893 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:32:43.578893 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:32:43.578893 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:32:43.578893 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:32:43.578893 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 06 13:32:43.578893 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:43.578893 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 261/521] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jul 6 13:32:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:43.579498 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec409ee-60a7-4f54-a9d3-c5c13e51616d None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:43.631623 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ac25e-8898-4331-8b24-3553ca84ce0f None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:43.634012 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:43.669309 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa3f090-0469-435b-9365-5e125b9bf0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:43.670207 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa3f090-0469-435b-9365-5e125b9bf0f4 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:43.670562 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa3f090-0469-435b-9365-5e125b9bf0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:43.670908 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa3f090-0469-435b-9365-5e125b9bf0f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:43.682971 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa3f090-0469-435b-9365-5e125b9bf0f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:43.683859 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec409ee-60a7-4f54-a9d3-c5c13e51616d None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:43.684830 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa3f090-0469-435b-9365-5e125b9bf0f4 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:43.703607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-1aa3f090-0469-435b-9365-5e125b9bf0f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:43.703607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:43.703607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:43.703607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:43.703607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:43.703607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:43.703607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:43.703607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:43.703607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:43.703607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:43.703607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:43.703607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:43.703607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:43.703607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:43.703607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:43.704687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:43.704687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:43.704687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:43.704687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:43.704687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:43.704687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:43.704687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:43.704687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:43.704687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:43.704687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:43.704687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:43.704687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:43.704687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:43.704687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:43.704687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:43.704687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:43.704687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:43.705980 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:43.705980 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:43.705980 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:43.705980 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:43.705980 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 261/522] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:43.706433 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa3f090-0469-435b-9365-5e125b9bf0f4 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:43.724908 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c13abe-5ef8-4bb3-a255-21931e01903a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:43.725596 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c13abe-5ef8-4bb3-a255-21931e01903a None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:43.725882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c13abe-5ef8-4bb3-a255-21931e01903a None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:43.726160 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c13abe-5ef8-4bb3-a255-21931e01903a None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:43.726887 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c13abe-5ef8-4bb3-a255-21931e01903a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:43.727191 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c13abe-5ef8-4bb3-a255-21931e01903a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:43.728549 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c13abe-5ef8-4bb3-a255-21931e01903a None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:43.744693 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 262/523] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:43.744693 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c13abe-5ef8-4bb3-a255-21931e01903a None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:43.791862 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:43.816900 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa3f090-0469-435b-9365-5e125b9bf0f4 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:43.847824 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c13abe-5ef8-4bb3-a255-21931e01903a None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:43.853042 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ea666-4563-4a1c-ba54-a2f07e9746a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:43.853809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ea666-4563-4a1c-ba54-a2f07e9746a7 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:43.854191 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ea666-4563-4a1c-ba54-a2f07e9746a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:43.854575 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ea666-4563-4a1c-ba54-a2f07e9746a7 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:43.856133 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ea666-4563-4a1c-ba54-a2f07e9746a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:43.856762 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ea666-4563-4a1c-ba54-a2f07e9746a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:43.859453 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ea666-4563-4a1c-ba54-a2f07e9746a7 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:43.879561 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 262/524] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:43.879561 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ea666-4563-4a1c-ba54-a2f07e9746a7 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:43.892813 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7e9a7-13bb-4ba7-9f1b-71af0f01fb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:43.893537 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7e9a7-13bb-4ba7-9f1b-71af0f01fb4d None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:43.893892 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7e9a7-13bb-4ba7-9f1b-71af0f01fb4d None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:43.894243 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7e9a7-13bb-4ba7-9f1b-71af0f01fb4d None admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:43.895168 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f7e9a7-13bb-4ba7-9f1b-71af0f01fb4d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:43.895555 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f7e9a7-13bb-4ba7-9f1b-71af0f01fb4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:43.897098 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f7e9a7-13bb-4ba7-9f1b-71af0f01fb4d None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:43.902831 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:43.920202 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 263/525] 10.222.0.20 () {56 vars in 1139 bytes} [Wed Jul 6 13:32:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:43.920202 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7e9a7-13bb-4ba7-9f1b-71af0f01fb4d None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:44.000243 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ea666-4563-4a1c-ba54-a2f07e9746a7 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:44.024918 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7e9a7-13bb-4ba7-9f1b-71af0f01fb4d None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:44.027375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:44.038570 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fdef3b-8734-4de6-902c-4a929323ff35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:44.039232 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fdef3b-8734-4de6-902c-4a929323ff35 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:44.039523 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fdef3b-8734-4de6-902c-4a929323ff35 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:44.039802 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fdef3b-8734-4de6-902c-4a929323ff35 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:44.040821 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fdef3b-8734-4de6-902c-4a929323ff35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:44.040821 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fdef3b-8734-4de6-902c-4a929323ff35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:44.042219 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fdef3b-8734-4de6-902c-4a929323ff35 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:44.055609 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 263/526] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:44.056331 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fdef3b-8734-4de6-902c-4a929323ff35 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:44.065883 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddea416-d1b5-4eaa-a2a4-8d3179ae1a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:44.066685 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddea416-d1b5-4eaa-a2a4-8d3179ae1a40 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:44.066964 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddea416-d1b5-4eaa-a2a4-8d3179ae1a40 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:44.067240 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddea416-d1b5-4eaa-a2a4-8d3179ae1a40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:44.075518 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddea416-d1b5-4eaa-a2a4-8d3179ae1a40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:44.076949 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddea416-d1b5-4eaa-a2a4-8d3179ae1a40 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:44.091623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-fddea416-d1b5-4eaa-a2a4-8d3179ae1a40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:32:44.091623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:44.091623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:44.091623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:44.091623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:44.091623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:44.091623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:44.091623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:44.091623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:44.091623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:44.091623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:44.091623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:44.091623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:32:44.091623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:32:44.091623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:32:44.092480 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:32:44.092480 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:44.092480 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:44.092480 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:44.092480 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:44.092480 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:44.092480 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:44.092480 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:44.092480 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:44.092480 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:44.092480 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:44.092480 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:44.092480 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:44.092480 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:44.092480 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:44.092480 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:44.092480 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:44.093353 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:32:44.093353 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:32:44.093353 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:32:44.093353 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:32:44.093353 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:32:44.093353 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:32:44.093353 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:32:44.093353 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:44.093353 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 264/527] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:32:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:44.093815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddea416-d1b5-4eaa-a2a4-8d3179ae1a40 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:44.147826 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:44.179834 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fdef3b-8734-4de6-902c-4a929323ff35 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:44.222092 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddea416-d1b5-4eaa-a2a4-8d3179ae1a40 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:44.235802 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edffcbd-7a1d-4dba-8fb6-14f43b22c5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:44.236624 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edffcbd-7a1d-4dba-8fb6-14f43b22c5e9 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:44.236951 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edffcbd-7a1d-4dba-8fb6-14f43b22c5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:44.237250 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edffcbd-7a1d-4dba-8fb6-14f43b22c5e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:44.244080 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edffcbd-7a1d-4dba-8fb6-14f43b22c5e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:44.246211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edffcbd-7a1d-4dba-8fb6-14f43b22c5e9 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:44.259561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7bc36d-2ce6-4ac3-b843-f72970796100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:44.260205 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7bc36d-2ce6-4ac3-b843-f72970796100 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:44.260484 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7bc36d-2ce6-4ac3-b843-f72970796100 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:44.260758 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7bc36d-2ce6-4ac3-b843-f72970796100 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:44.261274 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-6edffcbd-7a1d-4dba-8fb6-14f43b22c5e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:32:44.261274 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:44.261274 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:44.261274 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:44.261274 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:44.261274 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:44.261274 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:44.261274 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:44.261274 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:44.261274 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:44.261274 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:44.261274 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:44.261274 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:32:44.261274 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:32:44.261274 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:32:44.262467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:32:44.262467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:44.262467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:44.262467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:44.262467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:44.262467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:44.262467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:44.262467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:44.262467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:44.262467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:44.262467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:44.262467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:44.262467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:44.262467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:44.262467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:44.262467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:44.262467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:44.263319 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:32:44.263319 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:32:44.263319 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:32:44.263319 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:32:44.263319 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:32:44.263319 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:32:44.263319 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:32:44.263319 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:44.263319 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 264/528] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:32:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:44.263775 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edffcbd-7a1d-4dba-8fb6-14f43b22c5e9 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:44.306195 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:44.348483 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7bc36d-2ce6-4ac3-b843-f72970796100 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:44.349966 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7bc36d-2ce6-4ac3-b843-f72970796100 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:44.377553 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edffcbd-7a1d-4dba-8fb6-14f43b22c5e9 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:44.416187 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f213e89-3a70-4a70-9db8-f757c78a05b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:44.416909 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f213e89-3a70-4a70-9db8-f757c78a05b3 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:44.417692 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f213e89-3a70-4a70-9db8-f757c78a05b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:44.417692 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f213e89-3a70-4a70-9db8-f757c78a05b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:44.420630 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:44.426888 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f213e89-3a70-4a70-9db8-f757c78a05b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:44.428314 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f213e89-3a70-4a70-9db8-f757c78a05b3 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:44.443382 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-8f213e89-3a70-4a70-9db8-f757c78a05b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:32:44.443382 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:44.443382 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:44.443382 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:44.443382 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:44.443382 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:44.443382 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:44.443382 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:44.443382 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:44.443382 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:44.443382 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:44.443382 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:44.443382 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:32:44.443382 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:32:44.443382 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:32:44.449593 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:32:44.449593 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:44.449593 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:44.449593 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:44.449593 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:44.449593 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:44.449593 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:44.449593 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:44.449593 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:44.449593 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:44.449593 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:44.449593 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:44.449593 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:44.449593 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:44.449593 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:44.449593 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:44.449593 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:44.450629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:32:44.450629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:32:44.450629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:32:44.450629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:32:44.450629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:32:44.450629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:32:44.450629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:32:44.450629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:44.450629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 265/529] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:32:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:44.450629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f213e89-3a70-4a70-9db8-f757c78a05b3 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:44.466873 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 265/530] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:32:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:32:44.467693 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7bc36d-2ce6-4ac3-b843-f72970796100 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:44.542274 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:44.564497 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f213e89-3a70-4a70-9db8-f757c78a05b3 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:44.583506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7bc36d-2ce6-4ac3-b843-f72970796100 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:44.603258 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97acfa78-b6a1-4d81-ba05-0cae1a22f50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:44.603917 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97acfa78-b6a1-4d81-ba05-0cae1a22f50a None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:44.604207 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97acfa78-b6a1-4d81-ba05-0cae1a22f50a None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:44.604481 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97acfa78-b6a1-4d81-ba05-0cae1a22f50a None admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:44.606420 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97acfa78-b6a1-4d81-ba05-0cae1a22f50a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:44.606738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97acfa78-b6a1-4d81-ba05-0cae1a22f50a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:44.608179 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97acfa78-b6a1-4d81-ba05-0cae1a22f50a None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:44.618329 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022cbe84-3643-4617-b7fa-62b6f90f474a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:44.619223 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022cbe84-3643-4617-b7fa-62b6f90f474a None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:44.619223 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022cbe84-3643-4617-b7fa-62b6f90f474a None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:44.619519 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022cbe84-3643-4617-b7fa-62b6f90f474a None admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:44.620301 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022cbe84-3643-4617-b7fa-62b6f90f474a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:44.620657 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022cbe84-3643-4617-b7fa-62b6f90f474a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:44.622166 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022cbe84-3643-4617-b7fa-62b6f90f474a None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:44.632133 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 266/531] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:32:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:44.632133 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97acfa78-b6a1-4d81-ba05-0cae1a22f50a None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:44.640424 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 266/532] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:32:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:44.641949 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022cbe84-3643-4617-b7fa-62b6f90f474a None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:44.692106 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:44.739082 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97acfa78-b6a1-4d81-ba05-0cae1a22f50a None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:44.757454 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022cbe84-3643-4617-b7fa-62b6f90f474a None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:44.775936 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965036f-898a-47fe-9a6b-41b2ec9f86ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:44.776627 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965036f-898a-47fe-9a6b-41b2ec9f86ec None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:44.777465 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965036f-898a-47fe-9a6b-41b2ec9f86ec None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:44.777742 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965036f-898a-47fe-9a6b-41b2ec9f86ec None admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:44.778527 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9965036f-898a-47fe-9a6b-41b2ec9f86ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:44.778831 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9965036f-898a-47fe-9a6b-41b2ec9f86ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:44.780251 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9965036f-898a-47fe-9a6b-41b2ec9f86ec None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:44.783128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:44.794383 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 267/533] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:32:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:44.794787 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf519da-96ce-48af-a836-317dd3c05aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:44.795395 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965036f-898a-47fe-9a6b-41b2ec9f86ec None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:44.795936 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf519da-96ce-48af-a836-317dd3c05aab None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:44.796216 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf519da-96ce-48af-a836-317dd3c05aab None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:44.796491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf519da-96ce-48af-a836-317dd3c05aab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:44.809891 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf519da-96ce-48af-a836-317dd3c05aab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:44.811333 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf519da-96ce-48af-a836-317dd3c05aab None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:44.831261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-5cf519da-96ce-48af-a836-317dd3c05aab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:44.831261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:44.831261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:44.831261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:44.831261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:44.831261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:44.831261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:44.831261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:44.831261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:44.831261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:44.831261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:44.831261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:44.831261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:44.831261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:44.831261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:44.832186 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:44.832186 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:44.832186 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:44.832186 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:44.832186 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:44.832186 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:44.832186 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:44.832186 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:44.832186 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:44.832186 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:44.832186 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:44.832186 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:44.832186 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:44.832186 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:44.832186 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:44.832186 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:44.832186 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:44.833081 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:44.833081 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:44.833081 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:44.833081 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:44.833081 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 267/534] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:44.833081 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf519da-96ce-48af-a836-317dd3c05aab None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:44.918565 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965036f-898a-47fe-9a6b-41b2ec9f86ec None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:44.935721 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:44.947951 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf519da-96ce-48af-a836-317dd3c05aab None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:44.955640 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65ce494-8de9-4685-bb50-20e478241d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:44.956294 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ce494-8de9-4685-bb50-20e478241d91 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:44.956845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ce494-8de9-4685-bb50-20e478241d91 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:44.956845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ce494-8de9-4685-bb50-20e478241d91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:44.957691 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65ce494-8de9-4685-bb50-20e478241d91 None admin] RBAC: Adding query filter params (role.id=a6853d485f7942259b3646de3f386852, scope.project.id=bfe6c3d8da2743be9609184db2506303, user.id=4015c5ca2b8a438288820cd003f9a308) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:44.958001 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65ce494-8de9-4685-bb50-20e478241d91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:44.959427 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65ce494-8de9-4685-bb50-20e478241d91 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:44.973185 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 268/535] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:32:44 2022] GET /identity/v3/role_assignments?user.id=4015c5ca2b8a438288820cd003f9a308&scope.project.id=bfe6c3d8da2743be9609184db2506303&role.id=a6853d485f7942259b3646de3f386852 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:44.973185 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65ce494-8de9-4685-bb50-20e478241d91 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:45.003677 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19257b20-713c-4b1a-b622-a147cecf6f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:45.004594 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19257b20-713c-4b1a-b622-a147cecf6f57 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:45.004881 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19257b20-713c-4b1a-b622-a147cecf6f57 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:45.005178 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19257b20-713c-4b1a-b622-a147cecf6f57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:45.006727 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19257b20-713c-4b1a-b622-a147cecf6f57 None admin] RBAC: Adding query filter params (role.id=a6853d485f7942259b3646de3f386852, scope.project.id=bfe6c3d8da2743be9609184db2506303, user.id=6647eb1e8a1544ef9d3a9806f2d1fc51) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:45.007046 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19257b20-713c-4b1a-b622-a147cecf6f57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:45.009134 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19257b20-713c-4b1a-b622-a147cecf6f57 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:45.018432 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 268/536] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:32:44 2022] GET /identity/v3/role_assignments?user.id=6647eb1e8a1544ef9d3a9806f2d1fc51&scope.project.id=bfe6c3d8da2743be9609184db2506303&role.id=a6853d485f7942259b3646de3f386852 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:45.019295 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19257b20-713c-4b1a-b622-a147cecf6f57 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:45.073391 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:45.141037 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65ce494-8de9-4685-bb50-20e478241d91 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:45.177306 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f099c7-dccd-4537-8115-d8e8ae7d5e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:45.177306 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f099c7-dccd-4537-8115-d8e8ae7d5e40 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:45.177306 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f099c7-dccd-4537-8115-d8e8ae7d5e40 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:45.177306 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f099c7-dccd-4537-8115-d8e8ae7d5e40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:45.177727 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f099c7-dccd-4537-8115-d8e8ae7d5e40 None admin] RBAC: Adding query filter params (role.id=a6853d485f7942259b3646de3f386852, scope.project.id=bfe6c3d8da2743be9609184db2506303, user.id=5997058ea0fe456a89856e222722a5cc) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:45.177727 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f099c7-dccd-4537-8115-d8e8ae7d5e40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:45.177727 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f099c7-dccd-4537-8115-d8e8ae7d5e40 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:45.184275 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 269/537] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:32:44 2022] GET /identity/v3/role_assignments?user.id=5997058ea0fe456a89856e222722a5cc&scope.project.id=bfe6c3d8da2743be9609184db2506303&role.id=a6853d485f7942259b3646de3f386852 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:45.186586 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19257b20-713c-4b1a-b622-a147cecf6f57 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:45.260215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270c3c6-2f9f-4654-a055-8bc36e2d6dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:45.260900 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270c3c6-2f9f-4654-a055-8bc36e2d6dc0 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:45.261311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270c3c6-2f9f-4654-a055-8bc36e2d6dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:45.262363 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270c3c6-2f9f-4654-a055-8bc36e2d6dc0 None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:45.263102 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270c3c6-2f9f-4654-a055-8bc36e2d6dc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:45.263411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270c3c6-2f9f-4654-a055-8bc36e2d6dc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:45.264811 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270c3c6-2f9f-4654-a055-8bc36e2d6dc0 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:45.284182 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 269/538] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:45.285843 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f099c7-dccd-4537-8115-d8e8ae7d5e40 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:45.347530 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:45.410142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f099c7-dccd-4537-8115-d8e8ae7d5e40 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:45.447584 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01129343-b759-420f-8ebd-b537c60b66bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:45.448434 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01129343-b759-420f-8ebd-b537c60b66bc None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:45.449922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01129343-b759-420f-8ebd-b537c60b66bc None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:45.450290 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01129343-b759-420f-8ebd-b537c60b66bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:45.457665 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01129343-b759-420f-8ebd-b537c60b66bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:45.459139 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01129343-b759-420f-8ebd-b537c60b66bc None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:45.469529 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da877af5-ea01-417c-b85a-9d56a1fa786c None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:45.470018 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da877af5-ea01-417c-b85a-9d56a1fa786c None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:45.470093 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da877af5-ea01-417c-b85a-9d56a1fa786c None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:45.475352 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 270/539] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:32:45.476647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-01129343-b759-420f-8ebd-b537c60b66bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:32:45.476647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:45.476647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:45.476647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:45.476647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:45.476647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:45.476647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:45.476647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:45.476647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:45.476647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:45.476647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:45.476647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:45.476647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:32:45.476647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:32:45.476647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:32:45.477511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:32:45.477511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:45.477511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:45.477511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:45.477511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:45.477511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:45.477511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:45.477511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:45.477511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:45.477511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:45.477511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:45.477511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:45.477511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:45.477511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:45.477511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:45.477511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:45.477511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:45.478449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:32:45.478449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:32:45.478449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:32:45.478449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:32:45.478449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:32:45.478449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:32:45.478449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:32:45.478449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:45.478971 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d5ee9-6077-4ea8-b497-c01ff7bfdeea None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:45.478971 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d5ee9-6077-4ea8-b497-c01ff7bfdeea None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:45.478971 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d5ee9-6077-4ea8-b497-c01ff7bfdeea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:45.479163 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 270/540] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:32:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:45.490452 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01129343-b759-420f-8ebd-b537c60b66bc None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:45.505207 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-7a7d5ee9-6077-4ea8-b497-c01ff7bfdeea None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:45.506711 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-7a7d5ee9-6077-4ea8-b497-c01ff7bfdeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:45.632212 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 271/541] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 13:32:45.677094 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01129343-b759-420f-8ebd-b537c60b66bc None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:45.679925 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:45.695141 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7d5ee9-6077-4ea8-b497-c01ff7bfdeea None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:45.720887 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9854a1-d920-4906-b485-5cf665d58f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:45.721935 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9854a1-d920-4906-b485-5cf665d58f55 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:45.722417 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9854a1-d920-4906-b485-5cf665d58f55 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:45.722888 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9854a1-d920-4906-b485-5cf665d58f55 None admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:45.723966 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9854a1-d920-4906-b485-5cf665d58f55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:45.724473 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9854a1-d920-4906-b485-5cf665d58f55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:45.726144 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9854a1-d920-4906-b485-5cf665d58f55 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:45.744318 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 271/542] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:32:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:45.746547 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9854a1-d920-4906-b485-5cf665d58f55 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:45.781366 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:45.873012 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9854a1-d920-4906-b485-5cf665d58f55 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:45.915658 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4116a982-0326-4242-a1f3-b489b1da5ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:45.916236 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116a982-0326-4242-a1f3-b489b1da5ed7 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:45.916521 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116a982-0326-4242-a1f3-b489b1da5ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:45.916792 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116a982-0326-4242-a1f3-b489b1da5ed7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:45.917754 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4116a982-0326-4242-a1f3-b489b1da5ed7 None admin] RBAC: Adding query filter params (role.id=a6853d485f7942259b3646de3f386852, scope.project.id=bfe6c3d8da2743be9609184db2506303, user.id=b9f250f5dbc54d2e9293962ece848f6c) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:45.918066 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4116a982-0326-4242-a1f3-b489b1da5ed7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:45.919494 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4116a982-0326-4242-a1f3-b489b1da5ed7 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:45.935667 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 272/543] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:32:45 2022] GET /identity/v3/role_assignments?user.id=b9f250f5dbc54d2e9293962ece848f6c&scope.project.id=bfe6c3d8da2743be9609184db2506303&role.id=a6853d485f7942259b3646de3f386852 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:45.959387 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7d5ee9-6077-4ea8-b497-c01ff7bfdeea None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:46.014010 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e91f06-8d2c-42ee-8c93-704035ab96ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:46.014832 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e91f06-8d2c-42ee-8c93-704035ab96ad None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:46.015113 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e91f06-8d2c-42ee-8c93-704035ab96ad None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:46.015389 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e91f06-8d2c-42ee-8c93-704035ab96ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:46.031801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e91f06-8d2c-42ee-8c93-704035ab96ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:46.039752 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-73e91f06-8d2c-42ee-8c93-704035ab96ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:32:46.039752 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:46.039752 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:46.039752 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:46.039752 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:46.039752 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:46.039752 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:46.039752 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:46.039752 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:46.039752 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:46.039752 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:46.039752 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:46.039752 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:46.039752 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:46.039752 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:46.039752 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:46.040714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:46.040714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:46.040714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:46.040714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:46.040714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:46.040714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:46.040714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:46.040714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:46.040714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:46.040714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:46.040714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:46.040714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:46.040714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:46.040714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:46.040714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:46.040714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:46.041974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:46.041974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:46.041974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:46.041974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:46.041974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:46.041974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:46.041974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:46.041974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:46.041974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:46.041974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:46.041974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:32:46.041974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:46.041974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 272/544] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jul 6 13:32:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:32:46.050612 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4116a982-0326-4242-a1f3-b489b1da5ed7 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:46.083494 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:46.160720 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4116a982-0326-4242-a1f3-b489b1da5ed7 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:46.205400 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab635ac0-4c42-471a-99b5-852bfc938fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:46.206000 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab635ac0-4c42-471a-99b5-852bfc938fce None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:46.206283 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab635ac0-4c42-471a-99b5-852bfc938fce None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:46.206555 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab635ac0-4c42-471a-99b5-852bfc938fce None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:46.207315 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab635ac0-4c42-471a-99b5-852bfc938fce None admin] RBAC: Adding query filter params (name=service) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:46.207619 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab635ac0-4c42-471a-99b5-852bfc938fce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:46.209018 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab635ac0-4c42-471a-99b5-852bfc938fce None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:46.223180 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 273/545] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jul 6 13:32:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:46.228926 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e91f06-8d2c-42ee-8c93-704035ab96ad None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:46.334610 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:46.350952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e91f06-8d2c-42ee-8c93-704035ab96ad None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:46.402181 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81661399-58f2-42eb-93ae-8f1e60827e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:46.402980 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81661399-58f2-42eb-93ae-8f1e60827e75 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:46.403259 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81661399-58f2-42eb-93ae-8f1e60827e75 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:46.403532 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81661399-58f2-42eb-93ae-8f1e60827e75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:46.419404 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81661399-58f2-42eb-93ae-8f1e60827e75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:46.420895 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81661399-58f2-42eb-93ae-8f1e60827e75 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:46.444301 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-81661399-58f2-42eb-93ae-8f1e60827e75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:46.444301 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:46.444301 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:46.444301 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:46.444301 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:46.444301 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:46.444301 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:46.444301 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:46.444301 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:46.444301 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:46.444301 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:46.444301 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:46.444301 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:46.444301 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:46.444301 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:46.445224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:46.445224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:46.445224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:46.445224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:46.445224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:46.445224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:46.445224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:46.445224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:46.445224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:46.445224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:46.445224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:46.445224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:46.445224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:46.445224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:46.445224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:46.445224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:46.445224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:46.446522 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:46.446522 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:46.446522 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:46.446522 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:46.446522 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 273/546] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:46.453741 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab635ac0-4c42-471a-99b5-852bfc938fce None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:46.564635 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:46.575581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab635ac0-4c42-471a-99b5-852bfc938fce None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:46.610950 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f76b2-3c37-4df6-899b-21315ec9b960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:46.611611 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f76b2-3c37-4df6-899b-21315ec9b960 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:46.611894 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f76b2-3c37-4df6-899b-21315ec9b960 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:46.612171 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f76b2-3c37-4df6-899b-21315ec9b960 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:46.612886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f76b2-3c37-4df6-899b-21315ec9b960 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:46.613218 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f76b2-3c37-4df6-899b-21315ec9b960 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:46.614574 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f76b2-3c37-4df6-899b-21315ec9b960 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:46.628031 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 274/547] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:46.633511 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81661399-58f2-42eb-93ae-8f1e60827e75 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:46.769533 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81661399-58f2-42eb-93ae-8f1e60827e75 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:46.819103 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5c14c-50ae-4b25-901c-b81ee19ed1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:46.820095 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5c14c-50ae-4b25-901c-b81ee19ed1c5 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:46.820378 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5c14c-50ae-4b25-901c-b81ee19ed1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:46.820645 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5c14c-50ae-4b25-901c-b81ee19ed1c5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:46.830707 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5c14c-50ae-4b25-901c-b81ee19ed1c5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:46.832201 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5c14c-50ae-4b25-901c-b81ee19ed1c5 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:46.848145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-ddd5c14c-50ae-4b25-901c-b81ee19ed1c5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 06 13:32:46.848145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:46.848145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:46.848145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:46.848145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:46.848145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:46.848145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:46.848145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:46.848145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:46.848145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:46.848145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:46.848145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:46.848145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:32:46.848145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:32:46.848145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:32:46.848145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:32:46.849176 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:46.849176 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:46.849176 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:32:46.849176 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:32:46.849176 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:32:46.849176 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:32:46.849176 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:46.849176 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:46.849176 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:46.849176 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:46.849176 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:46.849176 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:46.849176 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:46.849176 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:46.849176 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:46.849176 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:46.849176 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:46.849176 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:46.850972 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:46.850972 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:46.850972 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:32:46.850972 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:32:46.850972 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:32:46.850972 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:32:46.850972 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:32:46.850972 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:32:46.850972 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 06 13:32:46.850972 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:46.850972 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 274/548] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:32:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:46.858258 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f76b2-3c37-4df6-899b-21315ec9b960 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:46.970333 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f76b2-3c37-4df6-899b-21315ec9b960 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:47.008522 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261658b-f829-4ddf-b006-d495efe2017f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:47.009214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261658b-f829-4ddf-b006-d495efe2017f None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:47.009498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261658b-f829-4ddf-b006-d495efe2017f None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:47.009770 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261658b-f829-4ddf-b006-d495efe2017f None admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:47.010604 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2261658b-f829-4ddf-b006-d495efe2017f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:47.011433 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2261658b-f829-4ddf-b006-d495efe2017f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:47.013084 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2261658b-f829-4ddf-b006-d495efe2017f None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:47.031556 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 275/549] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Jul 6 13:32:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:47.038071 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5c14c-50ae-4b25-901c-b81ee19ed1c5 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:47.041085 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:47.178625 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5c14c-50ae-4b25-901c-b81ee19ed1c5 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:47.229136 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165442e-72a7-44fc-92f6-2c224987af87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:47.229797 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165442e-72a7-44fc-92f6-2c224987af87 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:47.230613 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165442e-72a7-44fc-92f6-2c224987af87 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:47.230894 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165442e-72a7-44fc-92f6-2c224987af87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:47.338577 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165442e-72a7-44fc-92f6-2c224987af87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:47.341141 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165442e-72a7-44fc-92f6-2c224987af87 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:47.343087 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:47.470208 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 275/550] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:32:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:32:47.475817 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261658b-f829-4ddf-b006-d495efe2017f None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:47.496630 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:47.500973 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208ee20-0f72-429f-8eb5-c973fde98372 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:47.501310 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208ee20-0f72-429f-8eb5-c973fde98372 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:47.501678 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208ee20-0f72-429f-8eb5-c973fde98372 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:47.503192 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 276/551] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:47.514891 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53771a-2ab0-44fb-8d6c-acd296005633 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:47.515180 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53771a-2ab0-44fb-8d6c-acd296005633 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:47.515461 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53771a-2ab0-44fb-8d6c-acd296005633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:47.545983 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-da53771a-2ab0-44fb-8d6c-acd296005633 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:47.547173 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-da53771a-2ab0-44fb-8d6c-acd296005633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:47.602362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261658b-f829-4ddf-b006-d495efe2017f None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:47.644320 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c71d78-8dfb-4145-857e-b8ef8f8b34d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:47.645655 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c71d78-8dfb-4145-857e-b8ef8f8b34d9 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:47.646138 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c71d78-8dfb-4145-857e-b8ef8f8b34d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:47.646845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c71d78-8dfb-4145-857e-b8ef8f8b34d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:47.658861 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c71d78-8dfb-4145-857e-b8ef8f8b34d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:47.660844 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c71d78-8dfb-4145-857e-b8ef8f8b34d9 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:47.684933 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-e2c71d78-8dfb-4145-857e-b8ef8f8b34d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:47.684933 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:47.684933 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:47.684933 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:47.684933 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:47.684933 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:47.684933 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:47.684933 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:47.684933 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:47.684933 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:47.684933 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:47.684933 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:47.684933 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:47.684933 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:47.684933 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:47.685988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:47.685988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:47.685988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:47.685988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:47.685988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:47.685988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:47.685988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:47.685988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:47.685988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:47.685988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:47.685988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:47.685988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:47.685988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:47.685988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:47.685988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:47.685988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:47.685988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:47.686838 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:47.686838 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:47.686838 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:47.686838 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:47.689188 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 276/553] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:47.689307 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 277/553] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 13:32:47.706440 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53771a-2ab0-44fb-8d6c-acd296005633 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:47.716062 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c71d78-8dfb-4145-857e-b8ef8f8b34d9 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:47.780413 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:47.788197 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:47.827015 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c71d78-8dfb-4145-857e-b8ef8f8b34d9 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:47.838428 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53771a-2ab0-44fb-8d6c-acd296005633 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:47.864911 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5f891-7817-43f0-b378-744ae533b7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:47.866150 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5f891-7817-43f0-b378-744ae533b7a8 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:47.866643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5f891-7817-43f0-b378-744ae533b7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:47.867151 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5f891-7817-43f0-b378-744ae533b7a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:47.876067 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc8d6d-2a10-4957-ae61-f830c4572320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:47.876735 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc8d6d-2a10-4957-ae61-f830c4572320 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:47.877032 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc8d6d-2a10-4957-ae61-f830c4572320 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:47.877439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc8d6d-2a10-4957-ae61-f830c4572320 None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:47.878172 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc8d6d-2a10-4957-ae61-f830c4572320 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:47.878478 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc8d6d-2a10-4957-ae61-f830c4572320 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:47.879840 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc8d6d-2a10-4957-ae61-f830c4572320 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:47.880281 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa5f891-7817-43f0-b378-744ae533b7a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:47.889173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-dfa5f891-7817-43f0-b378-744ae533b7a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:32:47.889173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:47.889173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:47.889173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:47.889173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:47.889173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:47.889173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:47.889173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:47.889173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:47.889173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:47.889173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:47.889173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:47.889173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:47.889173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:47.889173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:47.889173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:47.890221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:47.890221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:47.890221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:47.890221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:47.890221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:47.890221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:47.890221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:47.890221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:47.890221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:47.890221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:47.890221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:47.890221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:47.890221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:47.890221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:47.890221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:47.890221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:47.891057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:47.891057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:47.891057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:47.891057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:47.891057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:47.891057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:47.891057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:47.891057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:47.891057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:47.891057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:47.891057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:32:47.891057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:47.898417 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 278/555] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:47.898417 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae5eba-79b2-44c7-af2c-0a83ab1b4be9 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:47.898647 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae5eba-79b2-44c7-af2c-0a83ab1b4be9 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:47.898736 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae5eba-79b2-44c7-af2c-0a83ab1b4be9 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:47.899050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 277/555] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jul 6 13:32:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:32:47.900138 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5f891-7817-43f0-b378-744ae533b7a8 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:47.900362 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 279/556] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:47.906690 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae5eba-79b2-44c7-af2c-0a83ab1b4be9 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:48.010590 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5f891-7817-43f0-b378-744ae533b7a8 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:48.019726 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae5eba-79b2-44c7-af2c-0a83ab1b4be9 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:48.032353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:48.036917 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:48.047541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f3b9b-64ca-4b87-8b76-2f5729e4693a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:48.048643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f3b9b-64ca-4b87-8b76-2f5729e4693a None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:48.049453 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f3b9b-64ca-4b87-8b76-2f5729e4693a None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:48.049956 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f3b9b-64ca-4b87-8b76-2f5729e4693a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:48.058943 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62ef35e-02dc-4745-8b11-bbd06c50978b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:48.059596 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ef35e-02dc-4745-8b11-bbd06c50978b None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:48.059927 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ef35e-02dc-4745-8b11-bbd06c50978b None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:48.060205 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ef35e-02dc-4745-8b11-bbd06c50978b None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:48.060308 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783f3b9b-64ca-4b87-8b76-2f5729e4693a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:48.061020 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62ef35e-02dc-4745-8b11-bbd06c50978b None admin] RBAC: Adding query filter params (name=service) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:48.061336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62ef35e-02dc-4745-8b11-bbd06c50978b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:48.062410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783f3b9b-64ca-4b87-8b76-2f5729e4693a None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:48.062800 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62ef35e-02dc-4745-8b11-bbd06c50978b None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:48.076624 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 280/557] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jul 6 13:32:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:48.076624 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f51825-88ae-42dd-8a94-830ba9f9ae4a None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:48.076624 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f51825-88ae-42dd-8a94-830ba9f9ae4a None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:48.076624 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f51825-88ae-42dd-8a94-830ba9f9ae4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:48.083901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-783f3b9b-64ca-4b87-8b76-2f5729e4693a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:32:48.083901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:48.083901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:48.083901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:48.083901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:48.083901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:48.083901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:48.083901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:48.083901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:48.083901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:48.083901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:48.083901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:48.083901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:32:48.083901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:32:48.083901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:32:48.085300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:32:48.085300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:48.085300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:48.085300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:48.085300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:48.085300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:48.085300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:48.085300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:48.085300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:48.085300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:48.085300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:48.085300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:48.085300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:48.085300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:48.085300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:48.085300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:48.085300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:48.086863 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:32:48.086863 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:32:48.086863 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:32:48.086863 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:32:48.086863 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:32:48.086863 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:32:48.086863 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:32:48.086863 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:48.090673 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 278/558] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:32:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:48.091596 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f3b9b-64ca-4b87-8b76-2f5729e4693a None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:48.107183 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-82f51825-88ae-42dd-8a94-830ba9f9ae4a None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:48.108866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-82f51825-88ae-42dd-8a94-830ba9f9ae4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:48.186807 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 281/559] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:48.187880 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f51825-88ae-42dd-8a94-830ba9f9ae4a None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:48.223565 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f3b9b-64ca-4b87-8b76-2f5729e4693a None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:48.258598 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bba12f-88fe-440b-9e4f-5d4c24e10550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:48.259378 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bba12f-88fe-440b-9e4f-5d4c24e10550 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:48.259660 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bba12f-88fe-440b-9e4f-5d4c24e10550 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:48.259930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bba12f-88fe-440b-9e4f-5d4c24e10550 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:48.270276 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bba12f-88fe-440b-9e4f-5d4c24e10550 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:48.271710 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bba12f-88fe-440b-9e4f-5d4c24e10550 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:48.289581 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:48.289721 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-c9bba12f-88fe-440b-9e4f-5d4c24e10550 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:48.289721 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:48.289721 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:48.289721 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:48.289721 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:48.289721 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:48.289721 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:48.289721 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:48.289721 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:48.289721 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:48.289721 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:48.289721 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:48.289721 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:48.289721 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:48.289721 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:48.290621 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:48.290621 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:48.290621 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:48.290621 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:48.290621 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:48.290621 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:48.290621 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:48.290621 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:48.290621 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:48.290621 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:48.290621 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:48.290621 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:48.290621 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:48.290621 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:48.290621 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:48.290621 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:48.290621 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:48.291508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:48.291508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:48.291508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:48.291508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:48.291508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 279/560] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:48.291779 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bba12f-88fe-440b-9e4f-5d4c24e10550 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:48.311016 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f51825-88ae-42dd-8a94-830ba9f9ae4a None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:48.323814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:48.343962 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b3c78c-6e55-45b6-a726-3e456397bc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:48.344672 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3c78c-6e55-45b6-a726-3e456397bc83 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:48.344954 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3c78c-6e55-45b6-a726-3e456397bc83 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:48.345257 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3c78c-6e55-45b6-a726-3e456397bc83 None admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:48.346043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b3c78c-6e55-45b6-a726-3e456397bc83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:48.346348 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b3c78c-6e55-45b6-a726-3e456397bc83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:48.347764 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b3c78c-6e55-45b6-a726-3e456397bc83 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:48.359834 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 282/561] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:32:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:48.361053 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87b86a-cc7c-4c3e-8f05-55e5fae7a180 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:48.361344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87b86a-cc7c-4c3e-8f05-55e5fae7a180 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:48.361617 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87b86a-cc7c-4c3e-8f05-55e5fae7a180 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:48.362856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 283/562] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:48.365947 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec8815-ba06-43c6-8b9b-e3ed5a57217d None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:48.366235 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec8815-ba06-43c6-8b9b-e3ed5a57217d None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:48.366510 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec8815-ba06-43c6-8b9b-e3ed5a57217d None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:48.367823 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 284/563] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:48.368508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec8815-ba06-43c6-8b9b-e3ed5a57217d None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:48.393296 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bba12f-88fe-440b-9e4f-5d4c24e10550 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:48.427585 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79511f-d266-439a-a424-0a439f46c671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:48.428372 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79511f-d266-439a-a424-0a439f46c671 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:48.428655 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79511f-d266-439a-a424-0a439f46c671 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:48.428929 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79511f-d266-439a-a424-0a439f46c671 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:48.439019 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79511f-d266-439a-a424-0a439f46c671 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:48.446936 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-1a79511f-d266-439a-a424-0a439f46c671 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:32:48.446936 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:48.446936 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:48.446936 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:48.446936 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:48.446936 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:48.446936 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:48.446936 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:48.446936 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:48.446936 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:48.446936 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:48.446936 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:48.446936 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:48.446936 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:48.446936 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:48.446936 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:48.447956 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:48.447956 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:48.447956 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:48.447956 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:48.447956 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:48.447956 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:48.447956 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:48.447956 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:48.447956 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:48.447956 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:48.447956 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:48.447956 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:48.447956 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:48.447956 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:48.447956 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:48.447956 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:48.449450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:48.449450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:48.449450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:48.449450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:48.449450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:48.449450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:48.449450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:48.449450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:48.449450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:48.449450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:48.449450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:32:48.449450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:48.449450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 280/564] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jul 6 13:32:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:48.450140 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79511f-d266-439a-a424-0a439f46c671 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:48.472753 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec8815-ba06-43c6-8b9b-e3ed5a57217d None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:48.506257 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46df42-500c-44ee-9eff-1174d3581001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:48.506989 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46df42-500c-44ee-9eff-1174d3581001 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:48.507337 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46df42-500c-44ee-9eff-1174d3581001 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:48.507624 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46df42-500c-44ee-9eff-1174d3581001 None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:48.508384 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e46df42-500c-44ee-9eff-1174d3581001 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:48.508697 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e46df42-500c-44ee-9eff-1174d3581001 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:48.510096 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e46df42-500c-44ee-9eff-1174d3581001 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:48.521061 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 285/565] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:48.522304 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1edf2-904a-4233-8872-97417f306747 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:48.522599 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1edf2-904a-4233-8872-97417f306747 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:48.522885 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1edf2-904a-4233-8872-97417f306747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:48.528498 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:48.539677 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-03a1edf2-904a-4233-8872-97417f306747 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:48.540793 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-03a1edf2-904a-4233-8872-97417f306747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:48.547543 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:48.559584 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79511f-d266-439a-a424-0a439f46c671 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:48.593169 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb084da-7407-4ccc-8d47-7c609a5c45e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:48.594125 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb084da-7407-4ccc-8d47-7c609a5c45e9 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:48.594125 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb084da-7407-4ccc-8d47-7c609a5c45e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:48.594359 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb084da-7407-4ccc-8d47-7c609a5c45e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:48.595235 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb084da-7407-4ccc-8d47-7c609a5c45e9 None admin] RBAC: Adding query filter params (role.id=a6853d485f7942259b3646de3f386852, scope.project.id=bfe6c3d8da2743be9609184db2506303, user.id=8c1bea05c37646b0a05fa8dda0367ef2) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:48.595493 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb084da-7407-4ccc-8d47-7c609a5c45e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:48.597185 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb084da-7407-4ccc-8d47-7c609a5c45e9 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:48.602219 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 281/566] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:32:48 2022] GET /identity/v3/role_assignments?user.id=8c1bea05c37646b0a05fa8dda0367ef2&scope.project.id=bfe6c3d8da2743be9609184db2506303&role.id=a6853d485f7942259b3646de3f386852 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:48.603372 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81185bf-35df-4ec6-82b3-08eade9f86bf None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:48.603659 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81185bf-35df-4ec6-82b3-08eade9f86bf None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:48.603930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81185bf-35df-4ec6-82b3-08eade9f86bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:48.618578 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 286/567] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:48.619784 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a1edf2-904a-4233-8872-97417f306747 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:48.624780 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-b81185bf-35df-4ec6-82b3-08eade9f86bf None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:48.626377 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-b81185bf-35df-4ec6-82b3-08eade9f86bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:48.715835 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 282/568] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:48.726704 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81185bf-35df-4ec6-82b3-08eade9f86bf None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:48.733029 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a1edf2-904a-4233-8872-97417f306747 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:48.771739 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094514e-4c92-425f-8a72-cbe9e7703cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:48.772592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094514e-4c92-425f-8a72-cbe9e7703cfc None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:48.772886 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094514e-4c92-425f-8a72-cbe9e7703cfc None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:48.773276 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094514e-4c92-425f-8a72-cbe9e7703cfc None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:48.774037 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d094514e-4c92-425f-8a72-cbe9e7703cfc None admin] RBAC: Adding query filter params (name=service) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:48.774347 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d094514e-4c92-425f-8a72-cbe9e7703cfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:48.775721 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d094514e-4c92-425f-8a72-cbe9e7703cfc None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:48.777795 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:48.790627 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 287/569] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jul 6 13:32:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:48.790627 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094514e-4c92-425f-8a72-cbe9e7703cfc None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:48.830258 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:48.846949 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81185bf-35df-4ec6-82b3-08eade9f86bf None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:48.882869 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e1c13-b4d4-4daf-b07c-6899b820e0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:48.883666 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e1c13-b4d4-4daf-b07c-6899b820e0a8 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:48.883946 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e1c13-b4d4-4daf-b07c-6899b820e0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:48.884220 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e1c13-b4d4-4daf-b07c-6899b820e0a8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:48.891832 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92e1c13-b4d4-4daf-b07c-6899b820e0a8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:48.893321 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92e1c13-b4d4-4daf-b07c-6899b820e0a8 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:48.899204 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094514e-4c92-425f-8a72-cbe9e7703cfc None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:48.909033 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-e92e1c13-b4d4-4daf-b07c-6899b820e0a8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 06 13:32:48.909033 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:48.909033 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:48.909033 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:48.909033 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:48.909033 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:48.909033 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:48.909033 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:48.909033 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:48.909033 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:48.909033 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:48.909033 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:48.909033 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:32:48.909033 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:32:48.909033 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:32:48.909033 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:32:48.910020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:48.910020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:48.910020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:32:48.910020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:32:48.910020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:32:48.910020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:32:48.910020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:48.910020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:48.910020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:48.910020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:48.910020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:48.910020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:48.910020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:48.910020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:48.910020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:48.910020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:48.910020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:48.910020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:48.910968 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:48.910968 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:48.910968 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:32:48.910968 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:32:48.910968 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:32:48.910968 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:32:48.910968 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:32:48.910968 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:32:48.910968 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 06 13:32:48.910968 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:48.910968 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 283/570] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Jul 6 13:32:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:48.911579 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e1c13-b4d4-4daf-b07c-6899b820e0a8 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:48.944164 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6ddcc-49e4-48f5-8ace-b6e248042fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:48.945639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6ddcc-49e4-48f5-8ace-b6e248042fe3 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:48.945932 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6ddcc-49e4-48f5-8ace-b6e248042fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:48.946208 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6ddcc-49e4-48f5-8ace-b6e248042fe3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:48.958277 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a6ddcc-49e4-48f5-8ace-b6e248042fe3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:48.966599 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-18a6ddcc-49e4-48f5-8ace-b6e248042fe3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:32:48.966599 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:48.966599 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:48.966599 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:48.966599 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:48.966599 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:48.966599 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:48.966599 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:48.966599 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:48.966599 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:48.966599 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:48.966599 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:48.966599 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:48.966599 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:48.966599 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:48.966599 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:48.967506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:48.967506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:48.967506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:48.967506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:48.967506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:48.967506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:48.967506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:48.967506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:48.967506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:48.967506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:48.967506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:48.967506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:48.967506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:48.967506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:48.967506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:48.967506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:48.968345 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:48.968345 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:48.968345 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:48.968345 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:48.968345 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:48.968345 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:48.968345 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:48.968345 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:48.968345 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:48.968345 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:48.968345 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:32:48.968345 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:48.972624 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 288/571] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jul 6 13:32:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:48.973468 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6ddcc-49e4-48f5-8ace-b6e248042fe3 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:49.018592 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e1c13-b4d4-4daf-b07c-6899b820e0a8 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:49.035273 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:49.056755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1a9f7-027b-4706-b2a3-30d049350938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:49.057658 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1a9f7-027b-4706-b2a3-30d049350938 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:49.058628 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1a9f7-027b-4706-b2a3-30d049350938 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:49.059286 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1a9f7-027b-4706-b2a3-30d049350938 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:49.071333 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb1a9f7-027b-4706-b2a3-30d049350938 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:49.079365 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-4bb1a9f7-027b-4706-b2a3-30d049350938 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:32:49.079365 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:49.079365 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:49.079365 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:49.079365 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:49.079365 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:49.079365 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:49.079365 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:49.079365 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:49.079365 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:49.079365 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:49.079365 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:49.079365 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:49.079365 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:49.079365 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:49.079365 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:49.080456 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:49.080456 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:49.080456 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:49.080456 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:49.080456 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:49.080456 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:49.080456 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:49.080456 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:49.080456 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:49.080456 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:49.080456 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:49.080456 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:49.080456 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:49.080456 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:49.080456 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:49.080456 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:49.081738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:49.081738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:49.081738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:49.081738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:49.081738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:49.081738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:49.081738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:49.081738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:49.081738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:49.081738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:49.081738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:32:49.081738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:49.081738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 284/572] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jul 6 13:32:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:49.081738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1a9f7-027b-4706-b2a3-30d049350938 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:49.084808 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6ddcc-49e4-48f5-8ace-b6e248042fe3 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:49.095108 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:49.121790 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182896b-8740-4db3-995b-1a615f710982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:49.122667 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182896b-8740-4db3-995b-1a615f710982 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:49.123021 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182896b-8740-4db3-995b-1a615f710982 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:49.123372 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182896b-8740-4db3-995b-1a615f710982 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:49.134083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6182896b-8740-4db3-995b-1a615f710982 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:49.135506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6182896b-8740-4db3-995b-1a615f710982 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:49.152983 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-6182896b-8740-4db3-995b-1a615f710982 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:49.152983 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:49.152983 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:49.152983 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:49.152983 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:49.152983 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:49.152983 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:49.152983 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:49.152983 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:49.152983 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:49.152983 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:49.152983 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:49.152983 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:49.152983 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:49.152983 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:49.153944 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:49.153944 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:49.153944 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:49.153944 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:49.153944 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:49.153944 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:49.153944 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:49.153944 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:49.153944 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:49.153944 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:49.153944 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:49.153944 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:49.153944 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:49.153944 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:49.153944 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:49.153944 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:49.153944 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:49.154818 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:49.154818 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:49.154818 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:49.154818 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:49.154818 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 289/573] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:49.155528 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182896b-8740-4db3-995b-1a615f710982 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:49.200683 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1a9f7-027b-4706-b2a3-30d049350938 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:49.235607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38eb7a9-2508-413c-91f5-c968142936ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:49.236639 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38eb7a9-2508-413c-91f5-c968142936ff None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:49.237230 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38eb7a9-2508-413c-91f5-c968142936ff None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:49.237757 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38eb7a9-2508-413c-91f5-c968142936ff None admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:49.238789 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38eb7a9-2508-413c-91f5-c968142936ff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:49.239296 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38eb7a9-2508-413c-91f5-c968142936ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:49.240930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38eb7a9-2508-413c-91f5-c968142936ff None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:49.261765 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 285/574] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Jul 6 13:32:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:49.261935 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182896b-8740-4db3-995b-1a615f710982 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:49.262965 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38eb7a9-2508-413c-91f5-c968142936ff None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:49.273319 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:49.306601 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f0619-24b4-43b1-bf75-ace4a3d02000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:49.307327 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f0619-24b4-43b1-bf75-ace4a3d02000 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:49.307559 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f0619-24b4-43b1-bf75-ace4a3d02000 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:49.308378 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f0619-24b4-43b1-bf75-ace4a3d02000 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:49.309176 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f0619-24b4-43b1-bf75-ace4a3d02000 None admin] RBAC: Adding query filter params (name=service) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:49.309488 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f0619-24b4-43b1-bf75-ace4a3d02000 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:49.310848 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f0619-24b4-43b1-bf75-ace4a3d02000 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:49.324066 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 290/575] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jul 6 13:32:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:49.324066 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f0619-24b4-43b1-bf75-ace4a3d02000 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:49.338619 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:49.380019 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38eb7a9-2508-413c-91f5-c968142936ff None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:49.415785 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b6337-026b-4f27-90a8-a79b5a3ca101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:49.416452 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b6337-026b-4f27-90a8-a79b5a3ca101 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:49.416743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b6337-026b-4f27-90a8-a79b5a3ca101 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:49.417791 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b6337-026b-4f27-90a8-a79b5a3ca101 None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:49.418318 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b6337-026b-4f27-90a8-a79b5a3ca101 None admin] RBAC: Adding query filter params (name=service) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:49.418629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b6337-026b-4f27-90a8-a79b5a3ca101 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:49.420009 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b6337-026b-4f27-90a8-a79b5a3ca101 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:49.432498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 286/576] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jul 6 13:32:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:49.433660 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f0619-24b4-43b1-bf75-ace4a3d02000 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:49.433774 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b6337-026b-4f27-90a8-a79b5a3ca101 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:49.468174 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53208c-fa55-4cb0-a3bc-0e8422a78820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:49.468836 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53208c-fa55-4cb0-a3bc-0e8422a78820 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:49.469153 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53208c-fa55-4cb0-a3bc-0e8422a78820 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:49.470048 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53208c-fa55-4cb0-a3bc-0e8422a78820 None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:49.470158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b53208c-fa55-4cb0-a3bc-0e8422a78820 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:49.470459 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b53208c-fa55-4cb0-a3bc-0e8422a78820 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:49.471812 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b53208c-fa55-4cb0-a3bc-0e8422a78820 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:49.484273 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 291/577] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:49.485095 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53208c-fa55-4cb0-a3bc-0e8422a78820 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:49.519751 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:49.545665 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b6337-026b-4f27-90a8-a79b5a3ca101 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:49.572647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:49.582663 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee88ab7a-d9f9-482d-ac2f-70a187c8b13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:49.583461 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88ab7a-d9f9-482d-ac2f-70a187c8b13e None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:49.583742 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88ab7a-d9f9-482d-ac2f-70a187c8b13e None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:49.584019 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88ab7a-d9f9-482d-ac2f-70a187c8b13e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:49.595391 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee88ab7a-d9f9-482d-ac2f-70a187c8b13e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:49.596316 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53208c-fa55-4cb0-a3bc-0e8422a78820 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:49.596831 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee88ab7a-d9f9-482d-ac2f-70a187c8b13e None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:49.614116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-ee88ab7a-d9f9-482d-ac2f-70a187c8b13e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:49.614116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:49.614116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:49.614116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:49.614116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:49.614116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:49.614116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:49.614116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:49.614116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:49.614116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:49.614116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:49.614116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:49.614116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:49.614116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:49.614116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:49.615170 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:49.615170 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:49.615170 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:49.615170 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:49.615170 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:49.615170 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:49.615170 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:49.615170 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:49.615170 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:49.615170 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:49.615170 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:49.615170 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:49.615170 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:49.615170 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:49.615170 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:49.615170 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:49.615170 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:49.616066 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:49.616066 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:49.616066 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:49.616066 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:49.616066 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 287/578] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:49.616516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee88ab7a-d9f9-482d-ac2f-70a187c8b13e None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:49.638374 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5444fce8-d567-4276-91cc-ef0d43b9a8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:49.639396 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5444fce8-d567-4276-91cc-ef0d43b9a8be None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:49.639822 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5444fce8-d567-4276-91cc-ef0d43b9a8be None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:49.640236 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5444fce8-d567-4276-91cc-ef0d43b9a8be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:49.652645 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5444fce8-d567-4276-91cc-ef0d43b9a8be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:49.655214 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5444fce8-d567-4276-91cc-ef0d43b9a8be None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:49.674489 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-5444fce8-d567-4276-91cc-ef0d43b9a8be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:49.674489 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:49.674489 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:49.674489 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:49.674489 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:49.674489 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:49.674489 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:49.674489 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:49.674489 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:49.674489 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:49.674489 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:49.674489 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:49.674489 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:49.674489 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:49.674489 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:49.675412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:49.675412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:49.675412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:49.675412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:49.675412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:49.675412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:49.675412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:49.675412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:49.675412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:49.675412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:49.675412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:49.675412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:49.675412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:49.675412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:49.675412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:49.675412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:49.675412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:49.676287 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:49.676287 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:49.676287 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:49.676287 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:49.676287 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 292/579] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:49.677301 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5444fce8-d567-4276-91cc-ef0d43b9a8be None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:49.731625 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee88ab7a-d9f9-482d-ac2f-70a187c8b13e None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:49.767366 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:49.770295 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef230540-f7a7-48ce-bea9-79d656895d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:49.771644 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef230540-f7a7-48ce-bea9-79d656895d4f None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:49.772142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef230540-f7a7-48ce-bea9-79d656895d4f None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:49.772421 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef230540-f7a7-48ce-bea9-79d656895d4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:49.784510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef230540-f7a7-48ce-bea9-79d656895d4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:49.785981 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef230540-f7a7-48ce-bea9-79d656895d4f None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:49.788495 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5444fce8-d567-4276-91cc-ef0d43b9a8be None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:49.804343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-ef230540-f7a7-48ce-bea9-79d656895d4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:49.804343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:49.804343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:49.804343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:49.804343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:49.804343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:49.804343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:49.804343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:49.804343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:49.804343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:49.804343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:49.804343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:49.804343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:49.804343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:49.804343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:49.805271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:49.805271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:49.805271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:49.805271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:49.805271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:49.805271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:49.805271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:49.805271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:49.805271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:49.805271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:49.805271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:49.805271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:49.805271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:49.805271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:49.805271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:49.805271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:49.805271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:49.806297 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:49.806297 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:49.806297 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:49.806297 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:49.808044 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 288/580] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:49.809306 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef230540-f7a7-48ce-bea9-79d656895d4f None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:49.827930 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a81a8c1-11a0-4b1b-a860-c3e10d2552c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:49.828658 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81a8c1-11a0-4b1b-a860-c3e10d2552c8 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:49.828944 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81a8c1-11a0-4b1b-a860-c3e10d2552c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:49.829255 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81a8c1-11a0-4b1b-a860-c3e10d2552c8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:49.836329 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a81a8c1-11a0-4b1b-a860-c3e10d2552c8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:49.837789 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a81a8c1-11a0-4b1b-a860-c3e10d2552c8 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:49.846455 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:49.853858 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-4a81a8c1-11a0-4b1b-a860-c3e10d2552c8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 06 13:32:49.853858 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:49.853858 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:49.853858 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:49.853858 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:49.853858 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:49.853858 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:49.853858 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:49.853858 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:49.853858 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:49.853858 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:49.853858 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:49.853858 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:32:49.853858 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:32:49.853858 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:32:49.853858 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:32:49.855357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:49.855357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:49.855357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:32:49.855357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:32:49.855357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:32:49.855357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:32:49.855357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:49.855357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:49.855357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:49.855357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:49.855357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:49.855357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:49.855357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:49.855357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:49.855357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:49.855357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:49.855357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:49.855357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:49.861560 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:49.861560 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:49.861560 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:32:49.861560 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:32:49.861560 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:32:49.861560 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:32:49.861560 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:32:49.861560 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:32:49.861560 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 06 13:32:49.861560 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:49.861560 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 293/581] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:32:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:49.861560 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a81a8c1-11a0-4b1b-a860-c3e10d2552c8 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:49.928886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef230540-f7a7-48ce-bea9-79d656895d4f None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:49.960160 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a81a8c1-11a0-4b1b-a860-c3e10d2552c8 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:49.965442 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b0cbb-4416-4476-8929-30652a564829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:49.966109 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b0cbb-4416-4476-8929-30652a564829 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:49.966391 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b0cbb-4416-4476-8929-30652a564829 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:49.966665 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b0cbb-4416-4476-8929-30652a564829 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:49.967384 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b0cbb-4416-4476-8929-30652a564829 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:49.967691 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b0cbb-4416-4476-8929-30652a564829 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:49.969527 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b0cbb-4416-4476-8929-30652a564829 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:49.982178 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 289/582] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:49.987139 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b0cbb-4416-4476-8929-30652a564829 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:49.995739 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd5809-f12a-4afa-b44a-e036122625dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:49.996400 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd5809-f12a-4afa-b44a-e036122625dc None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:49.996685 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd5809-f12a-4afa-b44a-e036122625dc None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:49.996962 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd5809-f12a-4afa-b44a-e036122625dc None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:49.997719 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd5809-f12a-4afa-b44a-e036122625dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:49.998189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd5809-f12a-4afa-b44a-e036122625dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:49.999600 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd5809-f12a-4afa-b44a-e036122625dc None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:50.015363 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 294/583] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:50.015363 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd5809-f12a-4afa-b44a-e036122625dc None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:50.015363 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:50.085972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:50.097072 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b0cbb-4416-4476-8929-30652a564829 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:50.122417 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd5809-f12a-4afa-b44a-e036122625dc None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:50.137782 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc9628-e8f1-40bc-8cd1-2b8f361002d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:50.138604 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc9628-e8f1-40bc-8cd1-2b8f361002d8 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:50.138895 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc9628-e8f1-40bc-8cd1-2b8f361002d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:50.139165 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc9628-e8f1-40bc-8cd1-2b8f361002d8 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:50.139902 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fc9628-e8f1-40bc-8cd1-2b8f361002d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:50.140206 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fc9628-e8f1-40bc-8cd1-2b8f361002d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:50.141619 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fc9628-e8f1-40bc-8cd1-2b8f361002d8 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:50.154749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 290/584] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:50.155470 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc9628-e8f1-40bc-8cd1-2b8f361002d8 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:50.162365 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04bdf1b-17dc-408c-8665-a4ad5c5a3b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:50.163037 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bdf1b-17dc-408c-8665-a4ad5c5a3b95 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:50.163319 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bdf1b-17dc-408c-8665-a4ad5c5a3b95 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:50.163601 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bdf1b-17dc-408c-8665-a4ad5c5a3b95 None admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:50.164450 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04bdf1b-17dc-408c-8665-a4ad5c5a3b95 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:50.164756 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04bdf1b-17dc-408c-8665-a4ad5c5a3b95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:50.166486 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04bdf1b-17dc-408c-8665-a4ad5c5a3b95 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:50.188623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 295/585] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Jul 6 13:32:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:50.188623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04bdf1b-17dc-408c-8665-a4ad5c5a3b95 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:50.249617 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:50.268851 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc9628-e8f1-40bc-8cd1-2b8f361002d8 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:50.297255 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04bdf1b-17dc-408c-8665-a4ad5c5a3b95 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:50.305960 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0866b824-3ff2-4a1c-884b-dd3b28c5ef63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:50.306958 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0866b824-3ff2-4a1c-884b-dd3b28c5ef63 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:50.307070 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0866b824-3ff2-4a1c-884b-dd3b28c5ef63 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:50.307308 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0866b824-3ff2-4a1c-884b-dd3b28c5ef63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:50.315893 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0866b824-3ff2-4a1c-884b-dd3b28c5ef63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:50.317388 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0866b824-3ff2-4a1c-884b-dd3b28c5ef63 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:50.331643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-0866b824-3ff2-4a1c-884b-dd3b28c5ef63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:32:50.331643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:50.331643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:50.331643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:50.331643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:50.331643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:50.331643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:50.331643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:50.331643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:50.331643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:50.331643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:50.331643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:50.331643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:32:50.331643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:32:50.331643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:32:50.332548 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:32:50.332548 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:50.332548 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:50.332548 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:50.332548 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:50.332548 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:50.332548 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:50.332548 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:50.332548 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:50.332548 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:50.332548 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:50.332548 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:50.332548 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:50.332548 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:50.332548 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:50.332548 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:50.332548 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:50.333518 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:32:50.333518 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:32:50.333518 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:32:50.333518 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:32:50.333518 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:32:50.333518 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:32:50.333518 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:32:50.333518 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:50.337866 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 291/586] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:32:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:50.338409 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0866b824-3ff2-4a1c-884b-dd3b28c5ef63 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:50.340510 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393791df-b09a-40ee-8822-7106f06299e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:50.341376 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393791df-b09a-40ee-8822-7106f06299e9 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:50.341801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393791df-b09a-40ee-8822-7106f06299e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:50.341964 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393791df-b09a-40ee-8822-7106f06299e9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:50.349581 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393791df-b09a-40ee-8822-7106f06299e9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:50.351020 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393791df-b09a-40ee-8822-7106f06299e9 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:50.351907 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:50.368347 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-393791df-b09a-40ee-8822-7106f06299e9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 06 13:32:50.368347 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:50.368347 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:50.368347 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:50.368347 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:50.368347 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:50.368347 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:50.368347 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:50.368347 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:50.368347 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:50.368347 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:50.368347 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:50.368347 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:32:50.368347 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:32:50.368347 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:32:50.368347 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:32:50.369359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:50.369359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:50.369359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:32:50.369359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:32:50.369359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:32:50.369359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:32:50.369359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:50.369359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:50.369359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:50.369359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:50.369359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:50.369359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:50.369359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:50.369359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:50.369359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:50.369359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:50.369359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:50.369359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:50.370564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:50.370564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:50.370564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:32:50.370564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:32:50.370564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:32:50.370564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:32:50.370564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:32:50.370564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:32:50.370564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 06 13:32:50.370564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:50.370564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 296/587] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:32:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:50.370564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393791df-b09a-40ee-8822-7106f06299e9 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:50.448847 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0866b824-3ff2-4a1c-884b-dd3b28c5ef63 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:50.483679 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f15c5-5634-474d-a06c-7a174f0127de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:50.484471 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f15c5-5634-474d-a06c-7a174f0127de None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:50.484754 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f15c5-5634-474d-a06c-7a174f0127de None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:50.484938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393791df-b09a-40ee-8822-7106f06299e9 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:50.485066 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f15c5-5634-474d-a06c-7a174f0127de None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:50.494653 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2f15c5-5634-474d-a06c-7a174f0127de None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:50.496580 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2f15c5-5634-474d-a06c-7a174f0127de None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:50.505684 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:50.512193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-6e2f15c5-5634-474d-a06c-7a174f0127de None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 06 13:32:50.512193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:50.512193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:50.512193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:50.512193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:50.512193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:50.512193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:50.512193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:50.512193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:50.512193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:50.512193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:50.512193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:50.512193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:32:50.512193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:32:50.512193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:32:50.512193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:32:50.513183 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:50.513183 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:50.513183 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:32:50.513183 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:32:50.513183 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:32:50.513183 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:32:50.513183 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:50.513183 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:50.513183 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:50.513183 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:50.513183 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:50.513183 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:50.513183 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:50.513183 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:50.513183 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:50.513183 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:50.513183 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:50.513183 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:50.514904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:50.514904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:50.514904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:32:50.514904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:32:50.514904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:32:50.514904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:32:50.514904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:32:50.514904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:32:50.514904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 06 13:32:50.514904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:50.514904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 292/588] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jul 6 13:32:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:50.514904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f15c5-5634-474d-a06c-7a174f0127de None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:50.524647 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa4d0e-7ed1-48b7-ae2a-1d07e45d669e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:50.525468 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa4d0e-7ed1-48b7-ae2a-1d07e45d669e None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:50.525750 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa4d0e-7ed1-48b7-ae2a-1d07e45d669e None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:50.526023 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa4d0e-7ed1-48b7-ae2a-1d07e45d669e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:50.538019 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa4d0e-7ed1-48b7-ae2a-1d07e45d669e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:50.539505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa4d0e-7ed1-48b7-ae2a-1d07e45d669e None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:50.562688 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-3ffa4d0e-7ed1-48b7-ae2a-1d07e45d669e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:50.562688 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:50.562688 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:50.562688 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:50.562688 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:50.562688 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:50.562688 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:50.562688 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:50.562688 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:50.562688 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:50.562688 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:50.562688 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:50.562688 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:50.562688 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:50.562688 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:50.563630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:50.563630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:50.563630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:50.563630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:50.563630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:50.563630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:50.563630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:50.563630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:50.563630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:50.563630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:50.563630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:50.563630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:50.563630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:50.563630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:50.563630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:50.563630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:50.563630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:50.564505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:50.564505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:50.564505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:50.564505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:50.567873 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 297/589] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:50.568702 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa4d0e-7ed1-48b7-ae2a-1d07e45d669e None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:50.594398 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:50.627196 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f15c5-5634-474d-a06c-7a174f0127de None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:50.666572 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ff892-b1eb-4eae-839a-a9df8f0ed74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:50.667928 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ff892-b1eb-4eae-839a-a9df8f0ed74e None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:50.668290 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ff892-b1eb-4eae-839a-a9df8f0ed74e None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:50.668562 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ff892-b1eb-4eae-839a-a9df8f0ed74e None admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:50.669382 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ff892-b1eb-4eae-839a-a9df8f0ed74e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:50.669691 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ff892-b1eb-4eae-839a-a9df8f0ed74e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:50.671110 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ff892-b1eb-4eae-839a-a9df8f0ed74e None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:50.674821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa4d0e-7ed1-48b7-ae2a-1d07e45d669e None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:50.683950 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 293/590] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:32:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:50.684682 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ff892-b1eb-4eae-839a-a9df8f0ed74e None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:50.708579 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e051f23e-e242-4300-a165-8ac4cb35436f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:50.709279 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051f23e-e242-4300-a165-8ac4cb35436f None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:50.709576 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051f23e-e242-4300-a165-8ac4cb35436f None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:50.709858 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051f23e-e242-4300-a165-8ac4cb35436f None admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:50.710707 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e051f23e-e242-4300-a165-8ac4cb35436f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:50.711115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e051f23e-e242-4300-a165-8ac4cb35436f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:50.712561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e051f23e-e242-4300-a165-8ac4cb35436f None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:50.730079 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 298/591] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Jul 6 13:32:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:50.730882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e051f23e-e242-4300-a165-8ac4cb35436f None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:50.760271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:50.784306 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ff892-b1eb-4eae-839a-a9df8f0ed74e None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:50.812163 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:50.818440 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afb8a2-3736-480a-be7f-ea87b48ef848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:50.819101 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afb8a2-3736-480a-be7f-ea87b48ef848 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:50.819383 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afb8a2-3736-480a-be7f-ea87b48ef848 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:50.819657 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afb8a2-3736-480a-be7f-ea87b48ef848 None admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:50.820483 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58afb8a2-3736-480a-be7f-ea87b48ef848 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:50.820791 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58afb8a2-3736-480a-be7f-ea87b48ef848 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:50.822257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58afb8a2-3736-480a-be7f-ea87b48ef848 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:50.829044 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e051f23e-e242-4300-a165-8ac4cb35436f None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:50.839715 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 294/592] 10.222.0.20 () {56 vars in 1139 bytes} [Wed Jul 6 13:32:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:50.840960 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585b800-26c1-449b-a092-450fe7cf2d75 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:50.841280 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585b800-26c1-449b-a092-450fe7cf2d75 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:50.841559 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585b800-26c1-449b-a092-450fe7cf2d75 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:50.846844 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 295/593] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:50.847467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7585b800-26c1-449b-a092-450fe7cf2d75 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:50.864291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e59fd49-2250-4ec2-946f-58bb77ecdb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:50.864956 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59fd49-2250-4ec2-946f-58bb77ecdb40 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:50.865274 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59fd49-2250-4ec2-946f-58bb77ecdb40 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:50.865560 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59fd49-2250-4ec2-946f-58bb77ecdb40 None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:50.866297 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e59fd49-2250-4ec2-946f-58bb77ecdb40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:50.866611 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e59fd49-2250-4ec2-946f-58bb77ecdb40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:50.867968 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e59fd49-2250-4ec2-946f-58bb77ecdb40 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:50.879230 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 299/594] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:50.880005 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e59fd49-2250-4ec2-946f-58bb77ecdb40 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:50.951561 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7585b800-26c1-449b-a092-450fe7cf2d75 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:50.968264 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:50.978290 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e59fd49-2250-4ec2-946f-58bb77ecdb40 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:50.986625 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ea6fc-ab63-44ff-8bb7-130e148101c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:50.987507 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ea6fc-ab63-44ff-8bb7-130e148101c3 None admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:50.987787 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ea6fc-ab63-44ff-8bb7-130e148101c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:50.988059 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ea6fc-ab63-44ff-8bb7-130e148101c3 None admin] PATH_INFO: `/v3/projects/bfe6c3d8da2743be9609184db2506303/users/6647eb1e8a1544ef9d3a9806f2d1fc51/roles/a6853d485f7942259b3646de3f386852` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:51.002097 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4ea6fc-ab63-44ff-8bb7-130e148101c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=bfe6c3d8da2743be9609184db2506303, user_id=6647eb1e8a1544ef9d3a9806f2d1fc51, role_id=a6853d485f7942259b3646de3f386852)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:51.003525 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4ea6fc-ab63-44ff-8bb7-130e148101c3 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:51.013827 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9978091-6985-44af-8e39-9371ac6c67a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:51.014618 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9978091-6985-44af-8e39-9371ac6c67a2 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:51.014901 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9978091-6985-44af-8e39-9371ac6c67a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:51.015178 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9978091-6985-44af-8e39-9371ac6c67a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:51.016445 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 296/595] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jul 6 13:32:50 2022] PUT /identity/v3/projects/bfe6c3d8da2743be9609184db2506303/users/6647eb1e8a1544ef9d3a9806f2d1fc51/roles/a6853d485f7942259b3646de3f386852 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:32:51.017255 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ea6fc-ab63-44ff-8bb7-130e148101c3 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:51.027001 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9978091-6985-44af-8e39-9371ac6c67a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:51.028957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9978091-6985-44af-8e39-9371ac6c67a2 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:51.047882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-c9978091-6985-44af-8e39-9371ac6c67a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:51.047882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:51.047882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:51.047882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:51.047882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:51.047882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:51.047882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:51.047882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:51.047882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:51.047882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:51.047882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:51.047882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:51.047882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:51.047882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:51.047882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:51.049158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:51.049158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:51.049158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:51.049158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:51.049158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:51.049158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:51.049158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:51.049158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:51.049158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:51.049158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:51.049158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:51.049158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:51.049158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:51.049158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:51.049158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:51.049158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:51.049158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:51.050072 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:51.050072 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:51.050072 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:51.050072 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:51.052506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 300/596] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:51.054190 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc616261-0579-4727-99c8-8284c9517718 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:51.054696 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc616261-0579-4727-99c8-8284c9517718 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:51.055200 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc616261-0579-4727-99c8-8284c9517718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:51.074206 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:51.076178 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-cc616261-0579-4727-99c8-8284c9517718 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:51.077474 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-cc616261-0579-4727-99c8-8284c9517718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:51.141934 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ea6fc-ab63-44ff-8bb7-130e148101c3 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:51.157478 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 301/597] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:51.158347 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc616261-0579-4727-99c8-8284c9517718 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:51.188742 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe81208-d0a4-444c-8941-651d3e9255e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:51.189935 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe81208-d0a4-444c-8941-651d3e9255e7 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:51.190392 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe81208-d0a4-444c-8941-651d3e9255e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:51.190672 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe81208-d0a4-444c-8941-651d3e9255e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:51.201989 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe81208-d0a4-444c-8941-651d3e9255e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:51.203438 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe81208-d0a4-444c-8941-651d3e9255e7 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:51.222733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-abe81208-d0a4-444c-8941-651d3e9255e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:51.222733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:51.222733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:51.222733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:51.222733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:51.222733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:51.222733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:51.222733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:51.222733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:51.222733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:51.222733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:51.222733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:51.222733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:51.222733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:51.222733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:51.223671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:51.223671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:51.223671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:51.223671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:51.223671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:51.223671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:51.223671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:51.223671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:51.223671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:51.223671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:51.223671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:51.223671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:51.223671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:51.223671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:51.223671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:51.223671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:51.223671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:51.225302 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:51.225302 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:51.225302 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:51.225302 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:51.225302 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 297/598] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:51.225302 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe81208-d0a4-444c-8941-651d3e9255e7 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:51.241720 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:51.272677 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc616261-0579-4727-99c8-8284c9517718 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:51.313697 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b73361-a9c0-448c-a75e-42689091080d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:51.314780 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b73361-a9c0-448c-a75e-42689091080d None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:51.315245 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b73361-a9c0-448c-a75e-42689091080d None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:51.315732 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b73361-a9c0-448c-a75e-42689091080d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:51.323166 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b73361-a9c0-448c-a75e-42689091080d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:51.325851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b73361-a9c0-448c-a75e-42689091080d None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:51.332327 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:51.340784 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-b2b73361-a9c0-448c-a75e-42689091080d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:32:51.340784 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:51.340784 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:51.340784 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:51.340784 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:51.340784 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:51.340784 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:51.340784 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:51.340784 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:51.340784 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:51.340784 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:51.340784 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:51.340784 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:32:51.340784 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:32:51.340784 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:32:51.341799 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:32:51.341799 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:51.341799 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:51.341799 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:51.341799 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:51.341799 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:51.341799 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:51.341799 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:51.341799 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:51.341799 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:51.341799 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:51.341799 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:51.341799 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:51.341799 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:51.341799 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:51.341799 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:51.341799 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:51.342670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:32:51.342670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:32:51.342670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:32:51.342670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:32:51.342670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:32:51.342670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:32:51.342670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:32:51.342670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:51.342670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 302/599] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:32:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:51.343159 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b73361-a9c0-448c-a75e-42689091080d None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:51.348492 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe81208-d0a4-444c-8941-651d3e9255e7 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:51.385790 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90059a17-406e-4d92-b443-7c1aee7f9fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:51.386579 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90059a17-406e-4d92-b443-7c1aee7f9fa1 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:51.386876 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90059a17-406e-4d92-b443-7c1aee7f9fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:51.387137 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90059a17-406e-4d92-b443-7c1aee7f9fa1 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:51.387865 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90059a17-406e-4d92-b443-7c1aee7f9fa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:51.388172 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90059a17-406e-4d92-b443-7c1aee7f9fa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:51.389574 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90059a17-406e-4d92-b443-7c1aee7f9fa1 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:51.406343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 298/600] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:51.406343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90059a17-406e-4d92-b443-7c1aee7f9fa1 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:51.459822 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b73361-a9c0-448c-a75e-42689091080d None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:51.492968 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:51.498348 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17412723-f77e-49e6-93de-4e75d213072d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:51.499141 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17412723-f77e-49e6-93de-4e75d213072d None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:51.499422 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17412723-f77e-49e6-93de-4e75d213072d None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:51.499697 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17412723-f77e-49e6-93de-4e75d213072d None admin] PATH_INFO: `/v3/roles/service` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:51.508548 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90059a17-406e-4d92-b443-7c1aee7f9fa1 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:51.510048 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17412723-f77e-49e6-93de-4e75d213072d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:51.518655 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-17412723-f77e-49e6-93de-4e75d213072d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:32:51.518655 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:51.518655 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:51.518655 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:51.518655 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:51.518655 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:51.518655 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:51.518655 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:51.518655 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:51.518655 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:51.518655 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:51.518655 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:51.518655 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:51.518655 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:51.518655 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:51.518655 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:51.519606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:51.519606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:51.519606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:51.519606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:51.519606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:51.519606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:51.519606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:51.519606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:51.519606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:51.519606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:51.519606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:51.519606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:51.519606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:51.519606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:51.519606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:51.519606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:51.520447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:51.520447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:51.520447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:51.520447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:51.520447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:51.520447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:51.520447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:51.520447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:51.520447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:51.520447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:51.520447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:32:51.520447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:51.520447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 303/601] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jul 6 13:32:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:51.521954 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17412723-f77e-49e6-93de-4e75d213072d None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:51.550477 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcef6d9-dab2-45b8-91a8-4d598a9aff09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:51.551279 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcef6d9-dab2-45b8-91a8-4d598a9aff09 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:51.551567 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcef6d9-dab2-45b8-91a8-4d598a9aff09 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:51.551839 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcef6d9-dab2-45b8-91a8-4d598a9aff09 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:51.552558 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcef6d9-dab2-45b8-91a8-4d598a9aff09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:51.552866 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcef6d9-dab2-45b8-91a8-4d598a9aff09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:51.554250 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcef6d9-dab2-45b8-91a8-4d598a9aff09 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:51.571449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 299/602] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:51.571449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcef6d9-dab2-45b8-91a8-4d598a9aff09 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:51.579994 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:51.639189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17412723-f77e-49e6-93de-4e75d213072d None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:51.674338 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056a383-83c1-4492-8684-9d0204661ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:51.674919 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056a383-83c1-4492-8684-9d0204661ec1 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:51.675199 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056a383-83c1-4492-8684-9d0204661ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:51.675476 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056a383-83c1-4492-8684-9d0204661ec1 None admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:51.676261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b056a383-83c1-4492-8684-9d0204661ec1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:51.676570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b056a383-83c1-4492-8684-9d0204661ec1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:51.678010 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b056a383-83c1-4492-8684-9d0204661ec1 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:51.678616 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcef6d9-dab2-45b8-91a8-4d598a9aff09 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:51.691527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 304/603] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:32:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:51.692196 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056a383-83c1-4492-8684-9d0204661ec1 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:51.720495 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49ee7d-0524-4d3c-81b1-41a7948a3b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:51.721382 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49ee7d-0524-4d3c-81b1-41a7948a3b71 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:51.721666 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49ee7d-0524-4d3c-81b1-41a7948a3b71 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:51.721951 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49ee7d-0524-4d3c-81b1-41a7948a3b71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:51.730934 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49ee7d-0524-4d3c-81b1-41a7948a3b71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:51.732436 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49ee7d-0524-4d3c-81b1-41a7948a3b71 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:51.746276 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:51.748247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-0a49ee7d-0524-4d3c-81b1-41a7948a3b71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:32:51.748247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:51.748247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:51.748247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:51.748247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:51.748247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:51.748247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:51.748247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:51.748247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:51.748247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:51.748247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:51.748247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:51.748247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:32:51.748247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:32:51.748247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:32:51.749336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:32:51.749336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:51.749336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:51.749336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:51.749336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:51.749336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:51.749336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:51.749336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:51.749336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:51.749336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:51.749336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:51.749336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:51.749336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:51.749336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:51.749336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:51.749336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:51.749336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:51.750497 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:32:51.750497 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:32:51.750497 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:32:51.750497 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:32:51.750497 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:32:51.750497 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:32:51.750497 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:32:51.750497 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:51.750497 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 300/604] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:32:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:51.751124 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49ee7d-0524-4d3c-81b1-41a7948a3b71 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:51.803400 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056a383-83c1-4492-8684-9d0204661ec1 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:51.822549 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:51.838826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062aed77-3a9e-4b08-bd6a-77998b36a761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:51.839996 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062aed77-3a9e-4b08-bd6a-77998b36a761 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:51.840270 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062aed77-3a9e-4b08-bd6a-77998b36a761 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:51.840593 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062aed77-3a9e-4b08-bd6a-77998b36a761 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:51.841377 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062aed77-3a9e-4b08-bd6a-77998b36a761 None admin] RBAC: Adding query filter params (name=service) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:51.841676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062aed77-3a9e-4b08-bd6a-77998b36a761 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:51.843027 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062aed77-3a9e-4b08-bd6a-77998b36a761 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:51.856291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 305/605] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jul 6 13:32:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:51.856291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062aed77-3a9e-4b08-bd6a-77998b36a761 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:51.863715 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49ee7d-0524-4d3c-81b1-41a7948a3b71 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:51.898813 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b87fb84-4081-43a5-bc1f-6501adaa9bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:51.899605 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87fb84-4081-43a5-bc1f-6501adaa9bfe None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:51.899885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87fb84-4081-43a5-bc1f-6501adaa9bfe None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:51.900158 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87fb84-4081-43a5-bc1f-6501adaa9bfe None admin] PATH_INFO: `/v3/projects/service` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:51.907725 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b87fb84-4081-43a5-bc1f-6501adaa9bfe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:51.909190 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b87fb84-4081-43a5-bc1f-6501adaa9bfe None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:51.924009 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-7b87fb84-4081-43a5-bc1f-6501adaa9bfe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:32:51.924009 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:51.924009 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:51.924009 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:51.924009 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:51.924009 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:51.924009 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:51.924009 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:51.924009 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:51.924009 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:51.924009 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:51.924009 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:51.924009 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:32:51.924009 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:32:51.924009 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:32:51.925142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:32:51.925142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:51.925142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:51.925142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:51.925142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:51.925142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:51.925142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:51.925142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:51.925142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:51.925142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:51.925142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:51.925142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:51.925142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:51.925142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:51.925142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:51.925142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:51.925142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:51.931071 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:32:51.931071 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:32:51.931071 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:32:51.931071 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:32:51.931071 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:32:51.931071 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:32:51.931071 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:32:51.931071 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:51.931071 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 301/606] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:32:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:51.931071 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b87fb84-4081-43a5-bc1f-6501adaa9bfe None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:51.972766 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062aed77-3a9e-4b08-bd6a-77998b36a761 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:51.986391 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:52.010572 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1f725-da22-427d-b57a-471b1a2422e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:52.012297 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1f725-da22-427d-b57a-471b1a2422e7 None admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:52.012689 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1f725-da22-427d-b57a-471b1a2422e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:52.013155 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1f725-da22-427d-b57a-471b1a2422e7 None admin] PATH_INFO: `/v3/projects/bfe6c3d8da2743be9609184db2506303/users/4015c5ca2b8a438288820cd003f9a308/roles/a6853d485f7942259b3646de3f386852` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:52.029212 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1f725-da22-427d-b57a-471b1a2422e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=bfe6c3d8da2743be9609184db2506303, user_id=4015c5ca2b8a438288820cd003f9a308, role_id=a6853d485f7942259b3646de3f386852)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:52.030628 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1f725-da22-427d-b57a-471b1a2422e7 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:52.039405 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b87fb84-4081-43a5-bc1f-6501adaa9bfe None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:52.043859 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 306/607] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jul 6 13:32:51 2022] PUT /identity/v3/projects/bfe6c3d8da2743be9609184db2506303/users/4015c5ca2b8a438288820cd003f9a308/roles/a6853d485f7942259b3646de3f386852 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:32:52.044424 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1f725-da22-427d-b57a-471b1a2422e7 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:52.101448 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:52.110227 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d923d27-a313-4856-a0c1-772b1c56ac00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:52.111156 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d923d27-a313-4856-a0c1-772b1c56ac00 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:52.111718 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d923d27-a313-4856-a0c1-772b1c56ac00 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:52.112000 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d923d27-a313-4856-a0c1-772b1c56ac00 None admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:52.112914 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d923d27-a313-4856-a0c1-772b1c56ac00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:52.113485 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d923d27-a313-4856-a0c1-772b1c56ac00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:52.115051 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d923d27-a313-4856-a0c1-772b1c56ac00 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:52.134774 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 302/608] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:32:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:52.136536 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d923d27-a313-4856-a0c1-772b1c56ac00 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:52.191655 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1f725-da22-427d-b57a-471b1a2422e7 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:52.231097 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775abb07-f18d-4e8e-8d4d-ae81171a5ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:52.231915 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775abb07-f18d-4e8e-8d4d-ae81171a5ece None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:52.232204 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775abb07-f18d-4e8e-8d4d-ae81171a5ece None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:52.232471 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775abb07-f18d-4e8e-8d4d-ae81171a5ece None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:52.243252 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775abb07-f18d-4e8e-8d4d-ae81171a5ece None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:52.244704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775abb07-f18d-4e8e-8d4d-ae81171a5ece None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:52.255563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d923d27-a313-4856-a0c1-772b1c56ac00 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:52.264697 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-775abb07-f18d-4e8e-8d4d-ae81171a5ece None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:52.264697 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:52.264697 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:52.264697 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:52.264697 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:52.264697 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:52.264697 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:52.264697 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:52.264697 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:52.264697 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:52.264697 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:52.264697 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:52.264697 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:52.264697 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:52.264697 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:52.265953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:52.265953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:52.265953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:52.265953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:52.265953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:52.265953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:52.265953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:52.265953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:52.265953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:52.265953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:52.265953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:52.265953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:52.265953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:52.265953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:52.265953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:52.265953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:52.265953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:52.267321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:52.267321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:52.267321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:52.267321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:52.267321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 307/609] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:52.267321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775abb07-f18d-4e8e-8d4d-ae81171a5ece None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:52.286077 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:52.309692 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcbd4ae-01ba-401a-b184-0b31474b789f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:52.310557 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcbd4ae-01ba-401a-b184-0b31474b789f None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:52.310850 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcbd4ae-01ba-401a-b184-0b31474b789f None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:52.311131 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcbd4ae-01ba-401a-b184-0b31474b789f None admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:52.312053 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcbd4ae-01ba-401a-b184-0b31474b789f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:52.312369 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcbd4ae-01ba-401a-b184-0b31474b789f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:52.313841 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcbd4ae-01ba-401a-b184-0b31474b789f None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:52.328358 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 303/610] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:32:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:52.329278 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcbd4ae-01ba-401a-b184-0b31474b789f None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:52.390574 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775abb07-f18d-4e8e-8d4d-ae81171a5ece None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:52.413395 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:52.436640 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e108b73-df9f-40d8-afa7-4143aa95ae5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:52.438231 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e108b73-df9f-40d8-afa7-4143aa95ae5a None admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:52.439118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e108b73-df9f-40d8-afa7-4143aa95ae5a None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:52.439609 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e108b73-df9f-40d8-afa7-4143aa95ae5a None admin] PATH_INFO: `/v3/projects/bfe6c3d8da2743be9609184db2506303/users/5997058ea0fe456a89856e222722a5cc/roles/a6853d485f7942259b3646de3f386852` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:52.445093 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcbd4ae-01ba-401a-b184-0b31474b789f None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:52.460158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e108b73-df9f-40d8-afa7-4143aa95ae5a None admin] RBAC: Authorizing `identity:create_grant(project_id=bfe6c3d8da2743be9609184db2506303, user_id=5997058ea0fe456a89856e222722a5cc, role_id=a6853d485f7942259b3646de3f386852)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:52.462703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e108b73-df9f-40d8-afa7-4143aa95ae5a None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:52.486721 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 308/611] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jul 6 13:32:52 2022] PUT /identity/v3/projects/bfe6c3d8da2743be9609184db2506303/users/5997058ea0fe456a89856e222722a5cc/roles/a6853d485f7942259b3646de3f386852 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:32:52.486721 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e108b73-df9f-40d8-afa7-4143aa95ae5a None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:52.498788 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd77aea-f49a-4f51-83c7-b967089478f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:52.499480 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd77aea-f49a-4f51-83c7-b967089478f5 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:52.501107 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd77aea-f49a-4f51-83c7-b967089478f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:52.501403 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd77aea-f49a-4f51-83c7-b967089478f5 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:52.503243 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd77aea-f49a-4f51-83c7-b967089478f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:52.505510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd77aea-f49a-4f51-83c7-b967089478f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:52.506649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd77aea-f49a-4f51-83c7-b967089478f5 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:52.533481 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 304/612] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:52.540060 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd77aea-f49a-4f51-83c7-b967089478f5 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:52.636550 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:52.698258 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e108b73-df9f-40d8-afa7-4143aa95ae5a None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:52.718362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd77aea-f49a-4f51-83c7-b967089478f5 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:52.732465 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:52.754346 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e62d259-6ed9-4a6d-8a62-a5e3781fcd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:52.756149 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62d259-6ed9-4a6d-8a62-a5e3781fcd9a None admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:52.756718 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62d259-6ed9-4a6d-8a62-a5e3781fcd9a None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:52.757314 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62d259-6ed9-4a6d-8a62-a5e3781fcd9a None admin] PATH_INFO: `/v3/projects/bfe6c3d8da2743be9609184db2506303/users/b9f250f5dbc54d2e9293962ece848f6c/roles/a6853d485f7942259b3646de3f386852` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:52.761324 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b90769-a53b-4b48-845b-de113cff8bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:52.762203 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b90769-a53b-4b48-845b-de113cff8bc4 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:52.762565 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b90769-a53b-4b48-845b-de113cff8bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:52.762914 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b90769-a53b-4b48-845b-de113cff8bc4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:52.773834 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b90769-a53b-4b48-845b-de113cff8bc4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:52.774927 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b90769-a53b-4b48-845b-de113cff8bc4 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:52.776924 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e62d259-6ed9-4a6d-8a62-a5e3781fcd9a None admin] RBAC: Authorizing `identity:create_grant(project_id=bfe6c3d8da2743be9609184db2506303, user_id=b9f250f5dbc54d2e9293962ece848f6c, role_id=a6853d485f7942259b3646de3f386852)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:52.779698 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e62d259-6ed9-4a6d-8a62-a5e3781fcd9a None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:52.794155 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-e5b90769-a53b-4b48-845b-de113cff8bc4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 06 13:32:52.794155 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:52.794155 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:52.794155 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:52.794155 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:52.794155 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:52.794155 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:52.794155 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:52.794155 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:52.794155 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:52.794155 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:52.794155 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:52.794155 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:32:52.794155 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:32:52.794155 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:32:52.794155 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:32:52.795241 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:52.795241 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:52.795241 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:32:52.795241 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:32:52.795241 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:32:52.795241 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:32:52.795241 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:52.795241 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:52.795241 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:52.795241 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:52.795241 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:52.795241 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:52.795241 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:52.795241 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:52.795241 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:52.795241 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:52.795241 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:52.795241 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:52.796481 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:52.796481 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:52.796481 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:32:52.796481 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:32:52.796481 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:32:52.796481 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:32:52.796481 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:32:52.796481 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:32:52.796481 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 06 13:32:52.796481 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:52.799276 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 305/613] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:32:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:32:52.808590 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b90769-a53b-4b48-845b-de113cff8bc4 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:52.812903 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 309/614] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jul 6 13:32:52 2022] PUT /identity/v3/projects/bfe6c3d8da2743be9609184db2506303/users/b9f250f5dbc54d2e9293962ece848f6c/roles/a6853d485f7942259b3646de3f386852 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:32:52.924781 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b90769-a53b-4b48-845b-de113cff8bc4 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:52.964700 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34b39a-f983-4cd3-afdc-7bd49e2ac4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:52.965649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34b39a-f983-4cd3-afdc-7bd49e2ac4db None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:52.966077 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34b39a-f983-4cd3-afdc-7bd49e2ac4db None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:52.966510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34b39a-f983-4cd3-afdc-7bd49e2ac4db None admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:52.967508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c34b39a-f983-4cd3-afdc-7bd49e2ac4db None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:52.967980 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c34b39a-f983-4cd3-afdc-7bd49e2ac4db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:52.969634 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c34b39a-f983-4cd3-afdc-7bd49e2ac4db None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:52.989850 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 306/615] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Jul 6 13:32:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:52.996191 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e62d259-6ed9-4a6d-8a62-a5e3781fcd9a None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:53.109438 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9dafb-961b-473c-a2fc-f794215e5d37 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:53.110171 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9dafb-961b-473c-a2fc-f794215e5d37 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:53.110703 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9dafb-961b-473c-a2fc-f794215e5d37 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:53.112559 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 307/616] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:32:53.117067 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e62d259-6ed9-4a6d-8a62-a5e3781fcd9a None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:53.118930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c9816-9942-49d4-bf74-29138cfeb392 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:53.119474 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c9816-9942-49d4-bf74-29138cfeb392 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:53.119958 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c9816-9942-49d4-bf74-29138cfeb392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:53.132058 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:53.142673 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-284c9816-9942-49d4-bf74-29138cfeb392 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:53.144212 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-284c9816-9942-49d4-bf74-29138cfeb392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:53.166748 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6db6f3-0a2e-4508-8d7c-c7c79e6e100b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:53.171210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6db6f3-0a2e-4508-8d7c-c7c79e6e100b None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:53.171210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6db6f3-0a2e-4508-8d7c-c7c79e6e100b None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:53.171210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6db6f3-0a2e-4508-8d7c-c7c79e6e100b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:53.178561 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:53.187349 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6db6f3-0a2e-4508-8d7c-c7c79e6e100b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:53.189540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6db6f3-0a2e-4508-8d7c-c7c79e6e100b None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:53.210712 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-4a6db6f3-0a2e-4508-8d7c-c7c79e6e100b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:53.210712 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:53.210712 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:53.210712 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:53.210712 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:53.210712 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:53.210712 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:53.210712 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:53.210712 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:53.210712 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:53.210712 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:53.210712 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:53.210712 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:53.210712 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:53.210712 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:53.211870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:53.211870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:53.211870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:53.211870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:53.211870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:53.211870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:53.211870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:53.211870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:53.211870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:53.211870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:53.211870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:53.211870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:53.211870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:53.211870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:53.211870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:53.211870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:53.211870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:53.213071 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:53.213071 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:53.213071 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:53.213071 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:53.213071 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 310/617] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:53.218648 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6db6f3-0a2e-4508-8d7c-c7c79e6e100b None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:53.233220 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 308/618] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 13:32:53.257310 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c9816-9942-49d4-bf74-29138cfeb392 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:53.335371 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6db6f3-0a2e-4508-8d7c-c7c79e6e100b None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:53.366067 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c9816-9942-49d4-bf74-29138cfeb392 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:53.373208 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf30ad0-c389-44cf-8e0b-6d5c424699ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:53.373887 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf30ad0-c389-44cf-8e0b-6d5c424699ff None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:53.374171 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf30ad0-c389-44cf-8e0b-6d5c424699ff None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:53.374451 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf30ad0-c389-44cf-8e0b-6d5c424699ff None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:53.375189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf30ad0-c389-44cf-8e0b-6d5c424699ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:53.375497 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf30ad0-c389-44cf-8e0b-6d5c424699ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:53.376868 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf30ad0-c389-44cf-8e0b-6d5c424699ff None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:53.390708 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 311/619] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:53.396025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf30ad0-c389-44cf-8e0b-6d5c424699ff None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:53.403662 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dbedf6-8715-4086-ad23-1e529780b588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:53.404465 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbedf6-8715-4086-ad23-1e529780b588 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:53.404798 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbedf6-8715-4086-ad23-1e529780b588 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:53.405100 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbedf6-8715-4086-ad23-1e529780b588 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:53.415615 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dbedf6-8715-4086-ad23-1e529780b588 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:53.423691 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-b9dbedf6-8715-4086-ad23-1e529780b588 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:32:53.423691 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:53.423691 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:53.423691 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:53.423691 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:53.423691 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:53.423691 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:53.423691 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:53.423691 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:53.423691 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:53.423691 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:53.423691 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:53.423691 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:53.423691 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:53.423691 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:53.423691 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:53.425052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:53.425052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:53.425052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:53.425052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:53.425052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:53.425052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:53.425052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:53.425052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:53.425052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:53.425052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:53.425052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:53.425052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:53.425052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:53.425052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:53.425052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:53.425052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:53.426304 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:53.426304 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:53.426304 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:53.426304 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:53.426304 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:53.426304 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:53.426304 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:53.426304 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:53.426304 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:53.426304 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:53.426304 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:32:53.426304 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:53.426304 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 309/620] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jul 6 13:32:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:53.427256 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:53.431303 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dbedf6-8715-4086-ad23-1e529780b588 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:53.465380 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:53.514980 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf30ad0-c389-44cf-8e0b-6d5c424699ff None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:53.536739 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dbedf6-8715-4086-ad23-1e529780b588 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:53.551309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b8cc6-41d2-43c8-b618-4e601e38db19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:53.552258 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b8cc6-41d2-43c8-b618-4e601e38db19 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:53.552548 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b8cc6-41d2-43c8-b618-4e601e38db19 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:53.552823 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b8cc6-41d2-43c8-b618-4e601e38db19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:53.576480 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206b8cc6-41d2-43c8-b618-4e601e38db19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:53.580661 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206b8cc6-41d2-43c8-b618-4e601e38db19 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:53.603018 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-206b8cc6-41d2-43c8-b618-4e601e38db19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:32:53.603018 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:53.603018 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:53.603018 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:53.603018 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:53.603018 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:53.603018 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:53.603018 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:53.603018 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:53.603018 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:53.603018 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:53.603018 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:53.603018 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:32:53.603018 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:32:53.603018 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:32:53.604151 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:32:53.604151 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:53.604151 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:53.604151 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:53.604151 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:53.604151 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:53.604151 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:53.604151 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:53.604151 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:53.604151 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:53.604151 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:53.604151 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:53.604151 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:53.604151 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:53.604151 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:53.604151 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:53.604151 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:53.605288 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:32:53.605288 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:32:53.605288 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:32:53.605288 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:32:53.605288 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:32:53.605288 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:32:53.605288 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:32:53.605288 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:53.611648 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d984bd2-2219-4000-a6e7-7388a69900ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:53.612328 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d984bd2-2219-4000-a6e7-7388a69900ec None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:53.612616 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d984bd2-2219-4000-a6e7-7388a69900ec None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:53.612893 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d984bd2-2219-4000-a6e7-7388a69900ec None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:53.613684 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d984bd2-2219-4000-a6e7-7388a69900ec None admin] RBAC: Adding query filter params (name=service) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:53.613995 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d984bd2-2219-4000-a6e7-7388a69900ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:53.615382 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d984bd2-2219-4000-a6e7-7388a69900ec None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:53.622041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 312/621] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:32:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:32:53.623774 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b8cc6-41d2-43c8-b618-4e601e38db19 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:53.633182 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 310/622] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jul 6 13:32:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:53.635009 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d984bd2-2219-4000-a6e7-7388a69900ec None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:53.727987 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:53.741570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b8cc6-41d2-43c8-b618-4e601e38db19 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:53.782777 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a237ede-8ad6-4bb5-9c3f-206bdbce2d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:53.783370 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a237ede-8ad6-4bb5-9c3f-206bdbce2d39 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:53.783653 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a237ede-8ad6-4bb5-9c3f-206bdbce2d39 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:53.783935 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a237ede-8ad6-4bb5-9c3f-206bdbce2d39 None admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:53.784736 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a237ede-8ad6-4bb5-9c3f-206bdbce2d39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:53.785084 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a237ede-8ad6-4bb5-9c3f-206bdbce2d39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:53.786653 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a237ede-8ad6-4bb5-9c3f-206bdbce2d39 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:53.800115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 313/623] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:32:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:53.805825 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a237ede-8ad6-4bb5-9c3f-206bdbce2d39 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:53.893367 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:53.937461 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d984bd2-2219-4000-a6e7-7388a69900ec None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:53.976731 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2dfcf-df55-444e-802b-0dde544c6f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:53.978568 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2dfcf-df55-444e-802b-0dde544c6f56 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:53.979176 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2dfcf-df55-444e-802b-0dde544c6f56 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:53.981179 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2dfcf-df55-444e-802b-0dde544c6f56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:53.987902 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a237ede-8ad6-4bb5-9c3f-206bdbce2d39 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:53.994455 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e2dfcf-df55-444e-802b-0dde544c6f56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:53.996676 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e2dfcf-df55-444e-802b-0dde544c6f56 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:54.015858 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-f6e2dfcf-df55-444e-802b-0dde544c6f56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:54.015858 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:54.015858 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:54.015858 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:54.015858 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:54.015858 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:54.015858 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:54.015858 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:54.015858 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:54.015858 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:54.015858 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:54.015858 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:54.015858 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:54.015858 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:54.015858 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:54.016810 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:54.016810 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:54.016810 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:54.016810 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:54.016810 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:54.016810 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:54.016810 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:54.016810 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:54.016810 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:54.016810 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:54.016810 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:54.016810 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:54.016810 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:54.016810 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:54.016810 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:54.016810 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:54.016810 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:54.017720 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:54.017720 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:54.017720 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:54.017720 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:54.019967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 311/624] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:54.025556 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2dfcf-df55-444e-802b-0dde544c6f56 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:54.082687 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac7ff6-a93b-48f5-90b2-7269b959ad10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:54.083524 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac7ff6-a93b-48f5-90b2-7269b959ad10 None admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:54.083809 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac7ff6-a93b-48f5-90b2-7269b959ad10 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:54.084086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac7ff6-a93b-48f5-90b2-7269b959ad10 None admin] PATH_INFO: `/v3/projects/bfe6c3d8da2743be9609184db2506303/users/8c1bea05c37646b0a05fa8dda0367ef2/roles/a6853d485f7942259b3646de3f386852` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:54.126941 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac7ff6-a93b-48f5-90b2-7269b959ad10 None admin] RBAC: Authorizing `identity:create_grant(project_id=bfe6c3d8da2743be9609184db2506303, user_id=8c1bea05c37646b0a05fa8dda0367ef2, role_id=a6853d485f7942259b3646de3f386852)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:54.128389 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac7ff6-a93b-48f5-90b2-7269b959ad10 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:54.131793 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2dfcf-df55-444e-802b-0dde544c6f56 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:54.148821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:54.160034 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 314/625] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jul 6 13:32:53 2022] PUT /identity/v3/projects/bfe6c3d8da2743be9609184db2506303/users/8c1bea05c37646b0a05fa8dda0367ef2/roles/a6853d485f7942259b3646de3f386852 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:32:54.174917 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c67da-b5b1-4156-9f58-72ff7bcb60b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:54.175932 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c67da-b5b1-4156-9f58-72ff7bcb60b8 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:54.176396 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c67da-b5b1-4156-9f58-72ff7bcb60b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:54.176893 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c67da-b5b1-4156-9f58-72ff7bcb60b8 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:54.177838 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c67da-b5b1-4156-9f58-72ff7bcb60b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:54.178340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c67da-b5b1-4156-9f58-72ff7bcb60b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:54.180006 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c67da-b5b1-4156-9f58-72ff7bcb60b8 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:54.195565 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 312/626] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:54.199333 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:54.206115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac7ff6-a93b-48f5-90b2-7269b959ad10 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:54.373926 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac7ff6-a93b-48f5-90b2-7269b959ad10 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:54.412857 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23759f01-3062-46ac-a82b-f125cb9cfbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:54.413733 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23759f01-3062-46ac-a82b-f125cb9cfbf0 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:54.414056 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23759f01-3062-46ac-a82b-f125cb9cfbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:54.414338 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23759f01-3062-46ac-a82b-f125cb9cfbf0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:54.422731 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23759f01-3062-46ac-a82b-f125cb9cfbf0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:54.424461 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23759f01-3062-46ac-a82b-f125cb9cfbf0 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:54.439703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-23759f01-3062-46ac-a82b-f125cb9cfbf0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 06 13:32:54.439703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:54.439703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:54.439703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:54.439703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:54.439703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:54.439703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:54.439703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:54.439703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:54.439703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:54.439703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:54.439703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:54.439703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:32:54.439703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:32:54.439703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:32:54.439703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:32:54.440681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:54.440681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:54.440681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:32:54.440681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:32:54.440681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:32:54.440681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:32:54.440681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:54.440681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:54.440681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:54.440681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:54.440681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:54.440681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:54.440681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:54.440681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:54.440681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:54.440681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:54.440681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:54.440681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:54.441679 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:54.441679 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:54.441679 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:32:54.441679 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:32:54.441679 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:32:54.441679 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:32:54.441679 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:32:54.441679 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:32:54.441679 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 06 13:32:54.441679 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:54.441679 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 315/627] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Jul 6 13:32:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:54.449305 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c67da-b5b1-4156-9f58-72ff7bcb60b8 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:54.572084 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c67da-b5b1-4156-9f58-72ff7bcb60b8 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:54.621482 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041b681-a1ce-4c4f-92e2-1f87c31d5d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:54.622185 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041b681-a1ce-4c4f-92e2-1f87c31d5d80 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:54.622472 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041b681-a1ce-4c4f-92e2-1f87c31d5d80 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:54.622751 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041b681-a1ce-4c4f-92e2-1f87c31d5d80 None admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:54.623605 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041b681-a1ce-4c4f-92e2-1f87c31d5d80 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:54.623915 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041b681-a1ce-4c4f-92e2-1f87c31d5d80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:54.625422 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041b681-a1ce-4c4f-92e2-1f87c31d5d80 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:54.644985 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 313/628] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Jul 6 13:32:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:54.650411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23759f01-3062-46ac-a82b-f125cb9cfbf0 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:54.653296 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781239d-63c9-4cb6-8ee8-3e85c2a68534 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:54.653592 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781239d-63c9-4cb6-8ee8-3e85c2a68534 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:54.653874 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781239d-63c9-4cb6-8ee8-3e85c2a68534 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:54.655250 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 314/629] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:54.665321 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94337298-dc36-46f1-8d6c-75e4307bfcec None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:54.665607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94337298-dc36-46f1-8d6c-75e4307bfcec None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:54.665886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94337298-dc36-46f1-8d6c-75e4307bfcec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:54.694330 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-94337298-dc36-46f1-8d6c-75e4307bfcec None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:54.695236 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-94337298-dc36-46f1-8d6c-75e4307bfcec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:54.718623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:54.762636 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23759f01-3062-46ac-a82b-f125cb9cfbf0 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:54.778317 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:54.809789 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3fb604-d0b2-40e1-8387-fe174abc0ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:54.811504 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fb604-d0b2-40e1-8387-fe174abc0ad1 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:54.812526 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fb604-d0b2-40e1-8387-fe174abc0ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:54.813186 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fb604-d0b2-40e1-8387-fe174abc0ad1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:54.819349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 315/630] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 13:32:54.889849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94337298-dc36-46f1-8d6c-75e4307bfcec None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:54.940643 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3fb604-d0b2-40e1-8387-fe174abc0ad1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:54.942458 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3fb604-d0b2-40e1-8387-fe174abc0ad1 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:55.000253 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94337298-dc36-46f1-8d6c-75e4307bfcec None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:55.031299 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:55.050367 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755ca5b-83a9-4121-a651-e5e3fe7e48b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:55.051183 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755ca5b-83a9-4121-a651-e5e3fe7e48b6 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:55.051469 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755ca5b-83a9-4121-a651-e5e3fe7e48b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:55.051744 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755ca5b-83a9-4121-a651-e5e3fe7e48b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:55.074884 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6755ca5b-83a9-4121-a651-e5e3fe7e48b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:55.077842 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 316/631] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:32:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:32:55.083151 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3fb604-d0b2-40e1-8387-fe174abc0ad1 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:55.091752 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-6755ca5b-83a9-4121-a651-e5e3fe7e48b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:32:55.091752 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:55.091752 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:55.091752 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:55.091752 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:55.091752 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:55.091752 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:55.091752 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:55.091752 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:55.091752 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:55.091752 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:55.091752 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:55.091752 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:55.091752 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:55.091752 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:55.091752 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:55.092710 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:55.092710 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:55.092710 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:55.092710 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:55.092710 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:55.092710 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:55.092710 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:55.092710 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:55.092710 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:55.092710 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:55.092710 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:55.092710 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:55.092710 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:55.092710 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:55.092710 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:55.092710 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:55.093604 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:55.093604 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:55.093604 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:55.093604 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:55.093604 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:55.093604 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:55.093604 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:55.093604 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:55.093604 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:55.093604 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:55.093604 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:32:55.093604 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:55.093604 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 316/632] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jul 6 13:32:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:32:55.098921 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755ca5b-83a9-4121-a651-e5e3fe7e48b6 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:55.176409 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:55.195756 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3fb604-d0b2-40e1-8387-fe174abc0ad1 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:55.232810 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1aa74-27de-40da-9c0b-d7427299a828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:55.233694 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1aa74-27de-40da-9c0b-d7427299a828 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:55.233979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1aa74-27de-40da-9c0b-d7427299a828 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:55.234254 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1aa74-27de-40da-9c0b-d7427299a828 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:55.243985 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755ca5b-83a9-4121-a651-e5e3fe7e48b6 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:55.245575 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d1aa74-27de-40da-9c0b-d7427299a828 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:55.247025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d1aa74-27de-40da-9c0b-d7427299a828 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:55.266770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-e6d1aa74-27de-40da-9c0b-d7427299a828 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:55.266770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:55.266770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:55.266770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:55.266770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:55.266770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:55.266770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:55.266770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:55.266770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:55.266770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:55.266770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:55.266770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:55.266770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:55.266770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:55.266770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:55.267721 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:55.267721 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:55.267721 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:55.267721 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:55.267721 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:55.267721 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:55.267721 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:55.267721 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:55.267721 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:55.267721 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:55.267721 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:55.267721 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:55.267721 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:55.267721 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:55.267721 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:55.267721 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:55.267721 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:55.268591 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:55.268591 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:55.268591 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:55.268591 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:55.268591 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 317/633] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:55.274069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471bc4d-2a0a-4f15-ad52-65231748b154 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:55.274353 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471bc4d-2a0a-4f15-ad52-65231748b154 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:55.274625 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471bc4d-2a0a-4f15-ad52-65231748b154 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:55.276139 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 318/634] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:55.277633 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471bc4d-2a0a-4f15-ad52-65231748b154 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:55.287172 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81f2ba-a67c-4222-9eb9-69fae7a73809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:55.288138 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81f2ba-a67c-4222-9eb9-69fae7a73809 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:55.288649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81f2ba-a67c-4222-9eb9-69fae7a73809 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:55.289202 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81f2ba-a67c-4222-9eb9-69fae7a73809 None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:55.290174 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81f2ba-a67c-4222-9eb9-69fae7a73809 None admin] RBAC: Adding query filter params (name=service) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:55.290694 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81f2ba-a67c-4222-9eb9-69fae7a73809 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:55.293408 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81f2ba-a67c-4222-9eb9-69fae7a73809 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:55.295744 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:55.303461 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 317/635] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jul 6 13:32:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:32:55.310584 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec9e7c-1a99-4314-bf5a-1a1f0e5020fb None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:55.311137 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec9e7c-1a99-4314-bf5a-1a1f0e5020fb None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:55.311641 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec9e7c-1a99-4314-bf5a-1a1f0e5020fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:55.329598 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-74ec9e7c-1a99-4314-bf5a-1a1f0e5020fb None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:55.331098 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-74ec9e7c-1a99-4314-bf5a-1a1f0e5020fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:55.400306 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471bc4d-2a0a-4f15-ad52-65231748b154 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:55.417332 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 318/636] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:55.418840 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec9e7c-1a99-4314-bf5a-1a1f0e5020fb None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:55.438540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42ff89-ed9b-4db9-bdba-73a463848d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:55.439125 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42ff89-ed9b-4db9-bdba-73a463848d8b None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:55.439407 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42ff89-ed9b-4db9-bdba-73a463848d8b None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:55.439682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42ff89-ed9b-4db9-bdba-73a463848d8b None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:55.440637 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42ff89-ed9b-4db9-bdba-73a463848d8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:55.440757 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42ff89-ed9b-4db9-bdba-73a463848d8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:55.442086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42ff89-ed9b-4db9-bdba-73a463848d8b None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:55.455403 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 319/637] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:55.460654 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42ff89-ed9b-4db9-bdba-73a463848d8b None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:55.461107 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:55.545859 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:55.550578 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec9e7c-1a99-4314-bf5a-1a1f0e5020fb None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:55.583605 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42ff89-ed9b-4db9-bdba-73a463848d8b None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:55.585065 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5134bd8-ba3a-4f07-8f5f-8b93a3dca229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:55.585861 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5134bd8-ba3a-4f07-8f5f-8b93a3dca229 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:55.586139 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5134bd8-ba3a-4f07-8f5f-8b93a3dca229 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:55.586412 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5134bd8-ba3a-4f07-8f5f-8b93a3dca229 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:55.598899 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5134bd8-ba3a-4f07-8f5f-8b93a3dca229 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:55.600366 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5134bd8-ba3a-4f07-8f5f-8b93a3dca229 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:55.619712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-a5134bd8-ba3a-4f07-8f5f-8b93a3dca229 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:55.619712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:55.619712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:55.619712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:55.619712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:55.619712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:55.619712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:55.619712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:55.619712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:55.619712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:55.619712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:55.619712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:55.619712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:55.619712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:55.619712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:55.621735 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:55.621735 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:55.621735 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:55.621735 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:55.621735 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:55.621735 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:55.621735 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:55.621735 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:55.621735 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:55.621735 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:55.621735 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:55.621735 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:55.621735 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:55.621735 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:55.621735 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:55.621735 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:55.621735 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:55.622650 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:55.622650 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:55.622650 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:55.622650 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:55.622650 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 319/638] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:55.622650 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5134bd8-ba3a-4f07-8f5f-8b93a3dca229 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:55.637077 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e854383-089b-4805-8ea9-b6eb66154f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:55.637886 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e854383-089b-4805-8ea9-b6eb66154f56 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:55.638171 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e854383-089b-4805-8ea9-b6eb66154f56 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:55.638449 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e854383-089b-4805-8ea9-b6eb66154f56 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:55.650458 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e854383-089b-4805-8ea9-b6eb66154f56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:55.659610 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-2e854383-089b-4805-8ea9-b6eb66154f56 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:32:55.659610 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:55.659610 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:55.659610 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:55.659610 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:55.659610 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:55.659610 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:55.659610 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:55.659610 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:55.659610 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:55.659610 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:55.659610 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:55.659610 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:55.659610 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:55.659610 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:55.659610 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:55.660717 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:55.660717 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:55.660717 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:55.660717 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:55.660717 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:55.660717 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:55.660717 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:55.660717 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:55.660717 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:55.660717 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:55.660717 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:55.660717 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:55.660717 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:55.660717 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:55.660717 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:55.660717 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:55.661593 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:55.661593 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:55.661593 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:55.661593 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:55.661593 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:55.661593 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:55.661593 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:55.661593 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:55.661593 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:55.661593 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:55.661593 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:32:55.661593 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:55.661593 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 320/639] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jul 6 13:32:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:55.662279 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006cf26-14ab-407b-a8c1-0cf954fd37d9 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:55.662354 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006cf26-14ab-407b-a8c1-0cf954fd37d9 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:55.662782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006cf26-14ab-407b-a8c1-0cf954fd37d9 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:55.668707 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 321/640] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:55.668707 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006cf26-14ab-407b-a8c1-0cf954fd37d9 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:55.718549 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:55.736476 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5134bd8-ba3a-4f07-8f5f-8b93a3dca229 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:55.771811 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26bd201-ebac-4c0e-9899-acf6db235e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:55.772798 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26bd201-ebac-4c0e-9899-acf6db235e1a None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:55.772988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26bd201-ebac-4c0e-9899-acf6db235e1a None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:55.773293 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26bd201-ebac-4c0e-9899-acf6db235e1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:55.781413 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26bd201-ebac-4c0e-9899-acf6db235e1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:55.782962 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26bd201-ebac-4c0e-9899-acf6db235e1a None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:55.785794 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006cf26-14ab-407b-a8c1-0cf954fd37d9 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:55.797743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-f26bd201-ebac-4c0e-9899-acf6db235e1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:32:55.797743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:55.797743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:55.797743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:55.797743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:55.797743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:55.797743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:55.797743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:55.797743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:55.797743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:55.797743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:55.797743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:55.797743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:32:55.797743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:32:55.797743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:32:55.799004 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:32:55.799004 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:55.799004 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:55.799004 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:55.799004 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:55.799004 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:55.799004 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:55.799004 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:55.799004 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:55.799004 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:55.799004 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:55.799004 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:55.799004 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:55.799004 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:55.799004 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:55.799004 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:55.799004 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:55.799910 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:32:55.799910 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:32:55.799910 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:32:55.799910 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:32:55.799910 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:32:55.799910 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:32:55.799910 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:32:55.799910 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:55.799910 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 320/641] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:32:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:55.800406 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26bd201-ebac-4c0e-9899-acf6db235e1a None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:55.820727 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:55.828143 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17ad85-328f-4650-b9fd-474302c4ccda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:55.828810 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17ad85-328f-4650-b9fd-474302c4ccda None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:55.829127 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17ad85-328f-4650-b9fd-474302c4ccda None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:55.829408 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17ad85-328f-4650-b9fd-474302c4ccda None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:55.830132 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17ad85-328f-4650-b9fd-474302c4ccda None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:55.830441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17ad85-328f-4650-b9fd-474302c4ccda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:55.831794 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17ad85-328f-4650-b9fd-474302c4ccda None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:55.847899 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 322/642] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:55.847899 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478967e-a1de-44ea-8e93-56e6cb932b99 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:55.847899 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478967e-a1de-44ea-8e93-56e6cb932b99 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:55.847899 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478967e-a1de-44ea-8e93-56e6cb932b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:55.862715 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-9478967e-a1de-44ea-8e93-56e6cb932b99 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:55.864351 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-9478967e-a1de-44ea-8e93-56e6cb932b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:55.918776 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26bd201-ebac-4c0e-9899-acf6db235e1a None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:55.943022 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 323/643] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:55.946209 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9478967e-a1de-44ea-8e93-56e6cb932b99 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:55.956902 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb15664-9ffd-4d43-8282-11e4d76a5315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:55.957727 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb15664-9ffd-4d43-8282-11e4d76a5315 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:55.958274 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb15664-9ffd-4d43-8282-11e4d76a5315 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:55.958486 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb15664-9ffd-4d43-8282-11e4d76a5315 None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:55.959391 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb15664-9ffd-4d43-8282-11e4d76a5315 None admin] RBAC: Adding query filter params (name=service) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:55.959800 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb15664-9ffd-4d43-8282-11e4d76a5315 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:55.961448 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb15664-9ffd-4d43-8282-11e4d76a5315 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:55.971230 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 321/644] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jul 6 13:32:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:55.972116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb15664-9ffd-4d43-8282-11e4d76a5315 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:55.982355 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:56.059769 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9478967e-a1de-44ea-8e93-56e6cb932b99 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:56.080993 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:56.090464 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb15664-9ffd-4d43-8282-11e4d76a5315 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:56.097457 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b3ed16-b722-4d8c-8ace-a33eb26a1fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:56.098133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3ed16-b722-4d8c-8ace-a33eb26a1fc8 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:56.098417 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3ed16-b722-4d8c-8ace-a33eb26a1fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:56.098695 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3ed16-b722-4d8c-8ace-a33eb26a1fc8 None admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:56.099495 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b3ed16-b722-4d8c-8ace-a33eb26a1fc8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:56.099804 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b3ed16-b722-4d8c-8ace-a33eb26a1fc8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:56.101276 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b3ed16-b722-4d8c-8ace-a33eb26a1fc8 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:56.113952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 324/645] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:32:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:56.114769 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b3ed16-b722-4d8c-8ace-a33eb26a1fc8 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:56.131838 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d8cc92-0d4f-4614-8ff7-1215e0fb217e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:56.132920 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8cc92-0d4f-4614-8ff7-1215e0fb217e None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:56.133385 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8cc92-0d4f-4614-8ff7-1215e0fb217e None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:56.133713 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8cc92-0d4f-4614-8ff7-1215e0fb217e None admin] PATH_INFO: `/v3/users/glance` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:56.142626 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d8cc92-0d4f-4614-8ff7-1215e0fb217e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:56.144079 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d8cc92-0d4f-4614-8ff7-1215e0fb217e None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:56.158649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-97d8cc92-0d4f-4614-8ff7-1215e0fb217e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 06 13:32:56.158649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:56.158649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:56.158649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:56.158649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:56.158649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:56.158649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:56.158649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:56.158649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:56.158649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:56.158649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:56.158649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:56.158649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:32:56.158649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:32:56.158649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:32:56.158649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:32:56.159640 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:56.159640 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:56.159640 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:32:56.159640 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:32:56.159640 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:32:56.159640 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:32:56.159640 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:56.159640 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:56.159640 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:56.159640 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:56.159640 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:56.159640 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:56.159640 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:56.159640 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:56.159640 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:56.159640 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:56.159640 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:56.159640 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:56.160738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:56.160738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:56.160738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:32:56.160738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:32:56.160738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:32:56.160738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:32:56.160738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:32:56.160738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:32:56.160738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 06 13:32:56.160738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:56.160738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 322/646] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:32:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:56.160738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d8cc92-0d4f-4614-8ff7-1215e0fb217e None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:56.227878 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b3ed16-b722-4d8c-8ace-a33eb26a1fc8 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:56.228924 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:56.266519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d8cc92-0d4f-4614-8ff7-1215e0fb217e None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:56.267742 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9510f3c-a133-4996-99cb-83217790fca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:56.268891 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9510f3c-a133-4996-99cb-83217790fca0 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:56.269195 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9510f3c-a133-4996-99cb-83217790fca0 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:56.269471 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9510f3c-a133-4996-99cb-83217790fca0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:56.283763 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9510f3c-a133-4996-99cb-83217790fca0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:56.295977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-c9510f3c-a133-4996-99cb-83217790fca0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:32:56.295977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:56.295977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:56.295977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:56.295977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:56.295977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:56.295977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:56.295977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:56.295977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:56.295977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:56.295977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:56.295977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:56.295977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:56.295977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:56.295977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:56.295977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:56.297143 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:56.297143 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:56.297143 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:56.297143 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:56.297143 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:56.297143 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:56.297143 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:56.297143 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:56.297143 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:56.297143 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:56.297143 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:56.297143 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:56.297143 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:56.297143 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:56.297143 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:56.297143 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:56.298080 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:56.298080 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:56.298080 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:56.298080 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:56.298080 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:56.298080 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:56.298080 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:56.298080 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:56.298080 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:56.298080 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:56.298080 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:32:56.298080 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:56.302709 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 325/647] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jul 6 13:32:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:56.303681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9510f3c-a133-4996-99cb-83217790fca0 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:56.311144 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5a401-cb1c-4070-af2f-bb1760acc5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:56.311945 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5a401-cb1c-4070-af2f-bb1760acc5c5 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:56.312231 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5a401-cb1c-4070-af2f-bb1760acc5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:56.312508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5a401-cb1c-4070-af2f-bb1760acc5c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:56.324792 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5a401-cb1c-4070-af2f-bb1760acc5c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:56.326293 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5a401-cb1c-4070-af2f-bb1760acc5c5 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:56.344429 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-20f5a401-cb1c-4070-af2f-bb1760acc5c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:56.344429 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:56.344429 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:56.344429 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:56.344429 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:56.344429 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:56.344429 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:56.344429 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:56.344429 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:56.344429 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:56.344429 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:56.344429 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:56.344429 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:56.344429 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:56.344429 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:56.345402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:56.345402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:56.345402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:56.345402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:56.345402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:56.345402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:56.345402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:56.345402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:56.345402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:56.345402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:56.345402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:56.345402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:56.345402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:56.345402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:56.345402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:56.345402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:56.345402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:56.346300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:56.346300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:56.346300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:56.346300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:56.346300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 323/648] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:56.347016 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5a401-cb1c-4070-af2f-bb1760acc5c5 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:56.354178 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:56.414148 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9510f3c-a133-4996-99cb-83217790fca0 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:56.455408 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6641c780-7d2d-4a50-a129-6bd435ecb266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:56.456075 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641c780-7d2d-4a50-a129-6bd435ecb266 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:56.456507 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641c780-7d2d-4a50-a129-6bd435ecb266 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:56.456910 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641c780-7d2d-4a50-a129-6bd435ecb266 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:56.457944 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6641c780-7d2d-4a50-a129-6bd435ecb266 None admin] RBAC: Adding query filter params (role.id=a6853d485f7942259b3646de3f386852, scope.project.id=bfe6c3d8da2743be9609184db2506303, user.id=6647eb1e8a1544ef9d3a9806f2d1fc51) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:56.458262 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6641c780-7d2d-4a50-a129-6bd435ecb266 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:56.460316 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6641c780-7d2d-4a50-a129-6bd435ecb266 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:56.477029 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5a401-cb1c-4070-af2f-bb1760acc5c5 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:56.477246 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 326/649] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:32:56 2022] GET /identity/v3/role_assignments?user.id=6647eb1e8a1544ef9d3a9806f2d1fc51&scope.project.id=bfe6c3d8da2743be9609184db2506303&role.id=a6853d485f7942259b3646de3f386852 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:56.477246 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6641c780-7d2d-4a50-a129-6bd435ecb266 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:56.491887 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:56.513412 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e32e27-140b-46cb-bb27-7312a67bac83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:56.514278 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e32e27-140b-46cb-bb27-7312a67bac83 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:56.514754 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e32e27-140b-46cb-bb27-7312a67bac83 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:56.515129 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e32e27-140b-46cb-bb27-7312a67bac83 None admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:56.516093 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e32e27-140b-46cb-bb27-7312a67bac83 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:56.516509 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e32e27-140b-46cb-bb27-7312a67bac83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:56.518378 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e32e27-140b-46cb-bb27-7312a67bac83 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:56.545045 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 324/650] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Jul 6 13:32:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:56.545045 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e32e27-140b-46cb-bb27-7312a67bac83 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:56.603819 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6641c780-7d2d-4a50-a129-6bd435ecb266 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:56.607133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:56.647889 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dec9f0-ad65-4832-b997-62dcf8a454a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:56.648690 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dec9f0-ad65-4832-b997-62dcf8a454a7 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:56.649067 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dec9f0-ad65-4832-b997-62dcf8a454a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:56.649454 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dec9f0-ad65-4832-b997-62dcf8a454a7 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:56.650335 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dec9f0-ad65-4832-b997-62dcf8a454a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:56.650715 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dec9f0-ad65-4832-b997-62dcf8a454a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:56.652923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dec9f0-ad65-4832-b997-62dcf8a454a7 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:56.661906 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 327/651] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jul 6 13:32:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:56.668359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dec9f0-ad65-4832-b997-62dcf8a454a7 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:56.668501 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e32e27-140b-46cb-bb27-7312a67bac83 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:56.706875 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127e358-396a-4137-8746-231042c07321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:56.707631 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127e358-396a-4137-8746-231042c07321 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:56.707984 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127e358-396a-4137-8746-231042c07321 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:56.708329 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127e358-396a-4137-8746-231042c07321 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:56.709213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7127e358-396a-4137-8746-231042c07321 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:56.709599 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7127e358-396a-4137-8746-231042c07321 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:56.711018 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7127e358-396a-4137-8746-231042c07321 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:56.725116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 325/652] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:56.726512 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5600c85-830c-4ff6-9fa3-0bb01fa66e0f None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:56.726921 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5600c85-830c-4ff6-9fa3-0bb01fa66e0f None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:56.727203 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5600c85-830c-4ff6-9fa3-0bb01fa66e0f None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:56.728712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 326/653] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:56.730595 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5600c85-830c-4ff6-9fa3-0bb01fa66e0f None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:56.770243 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:56.783046 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dec9f0-ad65-4832-b997-62dcf8a454a7 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:56.818155 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832a6e0-35e9-40cb-87c5-3131f8455eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:56.818812 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832a6e0-35e9-40cb-87c5-3131f8455eba None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:56.819144 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832a6e0-35e9-40cb-87c5-3131f8455eba None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:56.819423 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832a6e0-35e9-40cb-87c5-3131f8455eba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:56.850339 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5600c85-830c-4ff6-9fa3-0bb01fa66e0f None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:56.856456 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:56.887887 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06272ce0-bb65-4119-b535-62ae78646953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:56.888685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06272ce0-bb65-4119-b535-62ae78646953 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:56.888964 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06272ce0-bb65-4119-b535-62ae78646953 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:56.889336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06272ce0-bb65-4119-b535-62ae78646953 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:56.892824 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0832a6e0-35e9-40cb-87c5-3131f8455eba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:56.894376 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0832a6e0-35e9-40cb-87c5-3131f8455eba None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:56.900582 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06272ce0-bb65-4119-b535-62ae78646953 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:56.903281 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06272ce0-bb65-4119-b535-62ae78646953 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:56.922383 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-06272ce0-bb65-4119-b535-62ae78646953 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:56.922383 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:56.922383 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:56.922383 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:56.922383 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:56.922383 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:56.922383 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:56.922383 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:56.922383 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:56.922383 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:56.922383 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:56.922383 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:56.922383 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:56.922383 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:56.922383 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:56.923336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:56.923336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:56.923336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:56.923336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:56.923336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:56.923336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:56.923336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:56.923336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:56.923336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:56.923336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:56.923336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:56.923336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:56.923336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:56.923336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:56.923336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:56.923336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:56.923336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:56.924557 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:56.924557 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:56.924557 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:56.924557 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:56.924557 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 327/654] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:56.924845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06272ce0-bb65-4119-b535-62ae78646953 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:57.014249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 328/655] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:32:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:32:57.014249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd08b1e-a618-4318-978f-1b226f06ad37 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:57.014249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd08b1e-a618-4318-978f-1b226f06ad37 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:57.014249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd08b1e-a618-4318-978f-1b226f06ad37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:57.028137 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-cdd08b1e-a618-4318-978f-1b226f06ad37 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:57.029273 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-cdd08b1e-a618-4318-978f-1b226f06ad37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:57.035879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:57.050357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06272ce0-bb65-4119-b535-62ae78646953 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:57.091279 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a4729-7fd2-414e-b69b-96dcfdf0e459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:57.092079 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a4729-7fd2-414e-b69b-96dcfdf0e459 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:57.092363 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a4729-7fd2-414e-b69b-96dcfdf0e459 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:57.092667 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a4729-7fd2-414e-b69b-96dcfdf0e459 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:57.097703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:57.100545 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a4729-7fd2-414e-b69b-96dcfdf0e459 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:57.102065 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a4729-7fd2-414e-b69b-96dcfdf0e459 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:57.113043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 329/656] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:57.113043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd08b1e-a618-4318-978f-1b226f06ad37 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:57.116645 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-7d3a4729-7fd2-414e-b69b-96dcfdf0e459 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 06 13:32:57.116645 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:57.116645 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:57.116645 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:57.116645 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:57.116645 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:57.116645 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:57.116645 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:57.116645 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:57.116645 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:57.116645 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:57.116645 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:57.116645 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:32:57.116645 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:32:57.116645 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:32:57.116645 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:32:57.117583 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:57.117583 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:57.117583 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:32:57.117583 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:32:57.117583 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:32:57.117583 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:32:57.117583 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:57.117583 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:57.117583 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:57.117583 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:57.117583 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:57.117583 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:57.117583 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:57.117583 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:57.117583 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:57.117583 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:57.117583 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:57.117583 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:57.118508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:57.118508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:57.118508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:32:57.118508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:32:57.118508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:32:57.118508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:32:57.118508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:32:57.118508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:32:57.118508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 06 13:32:57.118508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:57.122776 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 328/657] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:32:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:57.123547 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a4729-7fd2-414e-b69b-96dcfdf0e459 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:57.223455 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd08b1e-a618-4318-978f-1b226f06ad37 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:57.228399 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a4729-7fd2-414e-b69b-96dcfdf0e459 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:57.261235 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf70040-6c15-4423-a3ff-29b8c6fa87f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:57.261902 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf70040-6c15-4423-a3ff-29b8c6fa87f8 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:57.262187 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf70040-6c15-4423-a3ff-29b8c6fa87f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:57.262464 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf70040-6c15-4423-a3ff-29b8c6fa87f8 None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:57.263249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf70040-6c15-4423-a3ff-29b8c6fa87f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:57.263560 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf70040-6c15-4423-a3ff-29b8c6fa87f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:57.264101 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96855c9-39e3-4431-be06-d8f2ebc5eab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:57.264891 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96855c9-39e3-4431-be06-d8f2ebc5eab1 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:57.264985 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf70040-6c15-4423-a3ff-29b8c6fa87f8 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:57.265242 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96855c9-39e3-4431-be06-d8f2ebc5eab1 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:57.265463 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96855c9-39e3-4431-be06-d8f2ebc5eab1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:57.277071 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 330/658] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:57.277964 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf70040-6c15-4423-a3ff-29b8c6fa87f8 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:57.283547 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96855c9-39e3-4431-be06-d8f2ebc5eab1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:57.284947 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96855c9-39e3-4431-be06-d8f2ebc5eab1 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:57.304743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-b96855c9-39e3-4431-be06-d8f2ebc5eab1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:57.304743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:57.304743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:57.304743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:57.304743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:57.304743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:57.304743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:57.304743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:57.304743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:57.304743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:57.304743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:57.304743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:57.304743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:57.304743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:57.304743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:57.305689 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:57.305689 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:57.305689 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:57.305689 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:57.305689 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:57.305689 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:57.305689 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:57.305689 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:57.305689 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:57.305689 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:57.305689 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:57.305689 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:57.305689 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:57.305689 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:57.305689 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:57.305689 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:57.305689 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:57.306569 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:57.306569 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:57.306569 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:57.306569 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:57.307233 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 329/659] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:57.308947 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96855c9-39e3-4431-be06-d8f2ebc5eab1 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:57.309995 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:57.356503 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:57.401938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf70040-6c15-4423-a3ff-29b8c6fa87f8 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:57.417626 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96855c9-39e3-4431-be06-d8f2ebc5eab1 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:57.437454 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf70597d-a5e6-4c4a-8c5e-75f7aa64e9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:57.438254 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70597d-a5e6-4c4a-8c5e-75f7aa64e9b1 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:57.438596 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70597d-a5e6-4c4a-8c5e-75f7aa64e9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:57.438934 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70597d-a5e6-4c4a-8c5e-75f7aa64e9b1 None admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:57.439828 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf70597d-a5e6-4c4a-8c5e-75f7aa64e9b1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:57.440198 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf70597d-a5e6-4c4a-8c5e-75f7aa64e9b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:57.441902 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf70597d-a5e6-4c4a-8c5e-75f7aa64e9b1 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:57.453594 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6698a0-57b9-43d2-ab6a-8b1e1f98af32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:57.454478 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6698a0-57b9-43d2-ab6a-8b1e1f98af32 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:57.454823 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6698a0-57b9-43d2-ab6a-8b1e1f98af32 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:57.455170 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6698a0-57b9-43d2-ab6a-8b1e1f98af32 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:57.463665 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 331/660] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Jul 6 13:32:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:57.463665 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf70597d-a5e6-4c4a-8c5e-75f7aa64e9b1 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:57.471079 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6698a0-57b9-43d2-ab6a-8b1e1f98af32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:57.478886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-5c6698a0-57b9-43d2-ab6a-8b1e1f98af32 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:32:57.478886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:57.478886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:57.478886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:57.478886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:57.478886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:57.478886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:57.478886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:57.478886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:57.478886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:57.478886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:57.478886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:57.478886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:57.478886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:57.478886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:57.478886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:57.479916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:57.479916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:57.479916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:57.479916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:57.479916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:57.479916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:57.479916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:57.479916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:57.479916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:57.479916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:57.479916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:57.479916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:57.479916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:57.479916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:57.479916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:57.479916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:57.480920 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:57.480920 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:57.480920 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:57.480920 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:57.480920 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:57.480920 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:57.480920 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:57.480920 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:57.480920 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:57.480920 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:57.480920 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:32:57.480920 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:57.481625 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 330/661] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jul 6 13:32:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:57.482298 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6698a0-57b9-43d2-ab6a-8b1e1f98af32 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:57.541964 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:57.579194 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf70597d-a5e6-4c4a-8c5e-75f7aa64e9b1 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:57.585684 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6698a0-57b9-43d2-ab6a-8b1e1f98af32 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:57.594577 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:57.617311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7c9d5-0d43-4e41-bd8b-4f1b225fa012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:57.618285 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7c9d5-0d43-4e41-bd8b-4f1b225fa012 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:57.618566 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7c9d5-0d43-4e41-bd8b-4f1b225fa012 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:57.618916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7c9d5-0d43-4e41-bd8b-4f1b225fa012 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:57.620754 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0d8fa-3542-4ab1-8bfc-78524a6f2c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:57.621509 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0d8fa-3542-4ab1-8bfc-78524a6f2c65 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:57.622297 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0d8fa-3542-4ab1-8bfc-78524a6f2c65 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:57.622297 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0d8fa-3542-4ab1-8bfc-78524a6f2c65 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:57.622828 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0d8fa-3542-4ab1-8bfc-78524a6f2c65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:57.623139 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0d8fa-3542-4ab1-8bfc-78524a6f2c65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:57.624490 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0d8fa-3542-4ab1-8bfc-78524a6f2c65 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:57.626988 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7c9d5-0d43-4e41-bd8b-4f1b225fa012 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:57.628396 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7c9d5-0d43-4e41-bd8b-4f1b225fa012 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:57.641634 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 331/662] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:57.641634 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0d8fa-3542-4ab1-8bfc-78524a6f2c65 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:57.643344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-47e7c9d5-0d43-4e41-bd8b-4f1b225fa012 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 06 13:32:57.643344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:57.643344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:57.643344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:57.643344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:57.643344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:57.643344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:57.643344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:57.643344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:57.643344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:57.643344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:57.643344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:57.643344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:32:57.643344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:32:57.643344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:32:57.643344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:32:57.644434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:57.644434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:57.644434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:32:57.644434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:32:57.644434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:32:57.644434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:32:57.644434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:57.644434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:57.644434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:57.644434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:57.644434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:57.644434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:57.644434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:57.644434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:57.644434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:57.644434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:57.644434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:57.644434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:57.646127 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:57.646127 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:57.646127 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:32:57.646127 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:32:57.646127 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:32:57.646127 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:32:57.646127 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:32:57.646127 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:32:57.646127 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 06 13:32:57.646127 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:57.648939 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 332/663] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jul 6 13:32:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:57.649643 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7c9d5-0d43-4e41-bd8b-4f1b225fa012 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:57.746939 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0d8fa-3542-4ab1-8bfc-78524a6f2c65 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:57.758594 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7c9d5-0d43-4e41-bd8b-4f1b225fa012 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:57.768208 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:57.783933 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc31145-6cf3-4b1a-a38f-0acae52acc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:57.784589 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc31145-6cf3-4b1a-a38f-0acae52acc2d None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:57.784870 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc31145-6cf3-4b1a-a38f-0acae52acc2d None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:57.785182 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc31145-6cf3-4b1a-a38f-0acae52acc2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:57.798393 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531509c4-6203-458e-9e3b-f75b0f252c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:57.799508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531509c4-6203-458e-9e3b-f75b0f252c03 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:57.800040 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531509c4-6203-458e-9e3b-f75b0f252c03 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:57.800578 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531509c4-6203-458e-9e3b-f75b0f252c03 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:57.801429 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531509c4-6203-458e-9e3b-f75b0f252c03 None admin] RBAC: Adding query filter params (name=service) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:57.801737 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531509c4-6203-458e-9e3b-f75b0f252c03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:57.803206 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531509c4-6203-458e-9e3b-f75b0f252c03 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:57.817019 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 333/664] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jul 6 13:32:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:57.817019 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531509c4-6203-458e-9e3b-f75b0f252c03 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:57.851230 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:57.860445 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc31145-6cf3-4b1a-a38f-0acae52acc2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:57.861939 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc31145-6cf3-4b1a-a38f-0acae52acc2d None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:57.919344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531509c4-6203-458e-9e3b-f75b0f252c03 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:57.954286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86cbd98-d1d7-4f81-a248-5fa5767c3c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:57.955181 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cbd98-d1d7-4f81-a248-5fa5767c3c58 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:57.955363 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cbd98-d1d7-4f81-a248-5fa5767c3c58 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:57.955639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cbd98-d1d7-4f81-a248-5fa5767c3c58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:57.967458 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86cbd98-d1d7-4f81-a248-5fa5767c3c58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:57.967625 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 332/665] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:32:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:32:57.967625 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc31145-6cf3-4b1a-a38f-0acae52acc2d None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:57.967812 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86cbd98-d1d7-4f81-a248-5fa5767c3c58 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:57.983090 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-b86cbd98-d1d7-4f81-a248-5fa5767c3c58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:32:57.983090 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:57.983090 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:57.983090 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:57.983090 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:57.983090 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:57.983090 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:57.983090 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:57.983090 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:57.983090 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:57.983090 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:57.983090 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:57.983090 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:32:57.983090 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:32:57.983090 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:32:57.984210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:32:57.984210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:57.984210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:57.984210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:57.984210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:57.984210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:57.984210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:57.984210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:57.984210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:57.984210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:57.984210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:57.984210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:57.984210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:57.984210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:57.984210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:57.984210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:57.984210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:57.985292 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:32:57.985292 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:32:57.985292 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:32:57.985292 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:32:57.985292 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:32:57.985292 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:32:57.985292 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:32:57.985292 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:57.985292 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 334/666] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:32:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:57.989847 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86cbd98-d1d7-4f81-a248-5fa5767c3c58 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:57.991462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:58.076034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc31145-6cf3-4b1a-a38f-0acae52acc2d None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:58.096519 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:58.104615 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86cbd98-d1d7-4f81-a248-5fa5767c3c58 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:58.110703 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ba35c-1c82-40fe-9da8-ac4aacd05732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:58.111477 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ba35c-1c82-40fe-9da8-ac4aacd05732 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:58.111855 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ba35c-1c82-40fe-9da8-ac4aacd05732 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:58.112245 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ba35c-1c82-40fe-9da8-ac4aacd05732 None admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:58.113216 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ba35c-1c82-40fe-9da8-ac4aacd05732 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:58.113621 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ba35c-1c82-40fe-9da8-ac4aacd05732 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:58.115127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ba35c-1c82-40fe-9da8-ac4aacd05732 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:58.132708 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 333/667] 10.222.0.20 () {56 vars in 1139 bytes} [Wed Jul 6 13:32:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:58.133685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ba35c-1c82-40fe-9da8-ac4aacd05732 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:58.142695 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846c57e-7b42-4ce6-a939-114fca584f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:58.143494 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846c57e-7b42-4ce6-a939-114fca584f06 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:58.143785 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846c57e-7b42-4ce6-a939-114fca584f06 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:58.144069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846c57e-7b42-4ce6-a939-114fca584f06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:58.155417 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b846c57e-7b42-4ce6-a939-114fca584f06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:58.156861 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b846c57e-7b42-4ce6-a939-114fca584f06 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:58.175312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-b846c57e-7b42-4ce6-a939-114fca584f06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:58.175312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:58.175312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:58.175312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:58.175312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:58.175312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:58.175312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:58.175312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:58.175312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:58.175312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:58.175312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:58.175312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:58.175312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:58.175312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:58.175312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:58.176413 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:58.176413 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:58.176413 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:58.176413 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:58.176413 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:58.176413 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:58.176413 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:58.176413 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:58.176413 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:58.176413 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:58.176413 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:58.176413 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:58.176413 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:58.176413 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:58.176413 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:58.176413 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:58.176413 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:58.177585 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:58.177585 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:58.177585 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:58.177585 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:58.181318 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 335/668] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:58.182170 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846c57e-7b42-4ce6-a939-114fca584f06 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:58.221670 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:58.246117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ba35c-1c82-40fe-9da8-ac4aacd05732 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:58.284760 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846c57e-7b42-4ce6-a939-114fca584f06 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:58.293725 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252ad79-c5aa-439f-a41f-05a6b8417ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:58.294529 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252ad79-c5aa-439f-a41f-05a6b8417ba9 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:58.294813 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252ad79-c5aa-439f-a41f-05a6b8417ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:58.295085 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252ad79-c5aa-439f-a41f-05a6b8417ba9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:58.308653 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6252ad79-c5aa-439f-a41f-05a6b8417ba9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:58.310737 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6252ad79-c5aa-439f-a41f-05a6b8417ba9 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:58.323695 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1bc10-14f4-4898-a7ee-b799166a8d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:58.324404 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1bc10-14f4-4898-a7ee-b799166a8d4f None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:58.324693 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1bc10-14f4-4898-a7ee-b799166a8d4f None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:58.324973 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1bc10-14f4-4898-a7ee-b799166a8d4f None admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:58.325796 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba1bc10-14f4-4898-a7ee-b799166a8d4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:58.326157 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba1bc10-14f4-4898-a7ee-b799166a8d4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:58.327520 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba1bc10-14f4-4898-a7ee-b799166a8d4f None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:58.328976 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-6252ad79-c5aa-439f-a41f-05a6b8417ba9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:58.328976 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:58.328976 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:58.328976 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:58.328976 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:58.328976 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:58.328976 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:58.328976 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:58.328976 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:58.328976 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:58.328976 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:58.328976 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:58.328976 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:58.328976 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:58.328976 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:58.330063 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:58.330063 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:58.330063 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:58.330063 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:58.330063 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:58.330063 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:58.330063 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:58.330063 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:58.330063 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:58.330063 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:58.330063 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:58.330063 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:58.330063 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:58.330063 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:58.330063 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:58.330063 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:58.330063 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:58.331214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:58.331214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:58.331214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:58.331214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:58.331214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 334/669] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:58.331516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252ad79-c5aa-439f-a41f-05a6b8417ba9 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:58.344086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 336/670] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:32:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:58.345265 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1bc10-14f4-4898-a7ee-b799166a8d4f None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:58.359022 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:58.440653 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252ad79-c5aa-439f-a41f-05a6b8417ba9 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:58.453558 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1bc10-14f4-4898-a7ee-b799166a8d4f None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:58.468643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:58.475280 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d73a258-85c6-4ee8-b640-da187baf95ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:58.476342 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73a258-85c6-4ee8-b640-da187baf95ad None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:58.476637 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73a258-85c6-4ee8-b640-da187baf95ad None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:58.476913 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73a258-85c6-4ee8-b640-da187baf95ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:58.490118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875c138-680c-47d6-9631-77e1716cfea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:58.490858 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875c138-680c-47d6-9631-77e1716cfea2 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:58.491209 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875c138-680c-47d6-9631-77e1716cfea2 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:58.491663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875c138-680c-47d6-9631-77e1716cfea2 None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:58.492562 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875c138-680c-47d6-9631-77e1716cfea2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:58.492943 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875c138-680c-47d6-9631-77e1716cfea2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:58.494383 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875c138-680c-47d6-9631-77e1716cfea2 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:58.508016 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 337/671] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:58.513494 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875c138-680c-47d6-9631-77e1716cfea2 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:58.547920 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d73a258-85c6-4ee8-b640-da187baf95ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:58.550458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d73a258-85c6-4ee8-b640-da187baf95ad None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:58.584149 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:58.618009 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875c138-680c-47d6-9631-77e1716cfea2 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:58.652636 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cdc804-a4d7-4d3c-a830-1614a603d41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:58.653332 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cdc804-a4d7-4d3c-a830-1614a603d41c None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:58.653629 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cdc804-a4d7-4d3c-a830-1614a603d41c None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:58.653882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cdc804-a4d7-4d3c-a830-1614a603d41c None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:58.654603 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cdc804-a4d7-4d3c-a830-1614a603d41c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:58.654905 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cdc804-a4d7-4d3c-a830-1614a603d41c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:58.656251 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cdc804-a4d7-4d3c-a830-1614a603d41c None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:58.663633 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 335/672] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:32:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:32:58.664357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d73a258-85c6-4ee8-b640-da187baf95ad None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:58.668624 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 338/673] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:58.669481 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cdc804-a4d7-4d3c-a830-1614a603d41c None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:58.699358 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:58.768085 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cdc804-a4d7-4d3c-a830-1614a603d41c None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:58.769381 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d73a258-85c6-4ee8-b640-da187baf95ad None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:58.794441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:58.803322 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d696b0-f82d-45da-a849-35385492e9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:58.804111 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d696b0-f82d-45da-a849-35385492e9c0 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:58.804221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c599d5df-6e74-46b6-9fe5-8400990d4951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:58.804399 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d696b0-f82d-45da-a849-35385492e9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:58.804681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d696b0-f82d-45da-a849-35385492e9c0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:58.804770 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599d5df-6e74-46b6-9fe5-8400990d4951 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:58.805038 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599d5df-6e74-46b6-9fe5-8400990d4951 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:58.805314 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599d5df-6e74-46b6-9fe5-8400990d4951 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:58.806145 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c599d5df-6e74-46b6-9fe5-8400990d4951 None admin] RBAC: Adding query filter params (role.id=a6853d485f7942259b3646de3f386852, scope.project.id=bfe6c3d8da2743be9609184db2506303, user.id=4015c5ca2b8a438288820cd003f9a308) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:58.806458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c599d5df-6e74-46b6-9fe5-8400990d4951 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:58.808048 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c599d5df-6e74-46b6-9fe5-8400990d4951 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:58.813028 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d696b0-f82d-45da-a849-35385492e9c0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:58.813707 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 336/674] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:32:58 2022] GET /identity/v3/role_assignments?user.id=4015c5ca2b8a438288820cd003f9a308&scope.project.id=bfe6c3d8da2743be9609184db2506303&role.id=a6853d485f7942259b3646de3f386852 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:58.814472 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c599d5df-6e74-46b6-9fe5-8400990d4951 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:58.814568 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d696b0-f82d-45da-a849-35385492e9c0 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:58.840375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-68d696b0-f82d-45da-a849-35385492e9c0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 06 13:32:58.840375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:58.840375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:58.840375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:58.840375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:58.840375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:58.840375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:58.840375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:58.840375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:58.840375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:58.840375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:58.840375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:58.840375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:32:58.840375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:32:58.840375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:32:58.840375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:32:58.841411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:58.841411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:58.841411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:32:58.841411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:32:58.841411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:32:58.841411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:32:58.841411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:58.841411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:58.841411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:58.841411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:58.841411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:58.841411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:58.841411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:58.841411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:58.841411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:58.841411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:58.841411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:58.841411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:58.842344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:58.842344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:58.842344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:32:58.842344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:32:58.842344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:32:58.842344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:32:58.842344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:32:58.842344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:32:58.842344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 06 13:32:58.842344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:58.845357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 339/675] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:32:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:58.846871 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d696b0-f82d-45da-a849-35385492e9c0 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:58.922632 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:58.930677 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c599d5df-6e74-46b6-9fe5-8400990d4951 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:58.960068 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d696b0-f82d-45da-a849-35385492e9c0 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:58.968673 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a5b5b-c472-4d9c-bb9a-7491edc7e8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:58.969506 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a5b5b-c472-4d9c-bb9a-7491edc7e8ef None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:58.969783 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a5b5b-c472-4d9c-bb9a-7491edc7e8ef None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:58.970053 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a5b5b-c472-4d9c-bb9a-7491edc7e8ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:58.982708 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a5b5b-c472-4d9c-bb9a-7491edc7e8ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:58.984172 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a5b5b-c472-4d9c-bb9a-7491edc7e8ef None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:59.001749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-352a5b5b-c472-4d9c-bb9a-7491edc7e8ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:59.001749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:59.001749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:59.001749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:59.001749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:59.001749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:59.001749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:59.001749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:59.001749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:59.001749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:59.001749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:59.001749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:59.001749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:32:59.001749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:32:59.001749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:32:59.002786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:32:59.002786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:59.002786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:59.002786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:59.002786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:59.002786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:59.002786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:59.002786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:59.002786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:59.002786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:59.002786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:59.002786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:59.002786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:59.002786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:59.002786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:59.002786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:59.002786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:59.003849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:32:59.003849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:32:59.003849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:32:59.003849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:32:59.003849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 337/676] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:59.005063 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da8303-cba8-460a-bf96-9ded89df8a6a None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:59.005063 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da8303-cba8-460a-bf96-9ded89df8a6a None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:59.005242 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da8303-cba8-460a-bf96-9ded89df8a6a None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:59.009402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 338/677] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:32:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:32:59.010957 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da8303-cba8-460a-bf96-9ded89df8a6a None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:59.014212 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a26c7-9e70-45fe-abfd-8cb86afe1b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:59.015021 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a26c7-9e70-45fe-abfd-8cb86afe1b45 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:59.015307 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a26c7-9e70-45fe-abfd-8cb86afe1b45 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:59.015591 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a26c7-9e70-45fe-abfd-8cb86afe1b45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:59.023356 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a26c7-9e70-45fe-abfd-8cb86afe1b45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:59.024819 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a26c7-9e70-45fe-abfd-8cb86afe1b45 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:59.040670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-b63a26c7-9e70-45fe-abfd-8cb86afe1b45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:32:59.040670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:59.040670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:59.040670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:59.040670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:59.040670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:59.040670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:59.040670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:59.040670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:59.040670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:59.040670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:59.040670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:59.040670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:32:59.040670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:32:59.040670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:32:59.042124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:32:59.042124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:59.042124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:59.042124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:59.042124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:59.042124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:59.042124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:59.042124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:59.042124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:59.042124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:59.042124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:59.042124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:59.042124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:59.042124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:59.042124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:59.042124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:59.042124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:59.043766 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:32:59.043766 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:32:59.043766 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:32:59.043766 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:32:59.043766 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:32:59.043766 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:32:59.043766 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:32:59.043766 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:59.043766 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 340/678] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:32:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:59.044326 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88ed32-2a4f-4be8-ae74-0eea736f39fc None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:59.044326 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88ed32-2a4f-4be8-ae74-0eea736f39fc None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:59.044918 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88ed32-2a4f-4be8-ae74-0eea736f39fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:59.063317 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-9d88ed32-2a4f-4be8-ae74-0eea736f39fc None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:32:59.064614 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-9d88ed32-2a4f-4be8-ae74-0eea736f39fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:32:59.111207 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:59.126126 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da8303-cba8-460a-bf96-9ded89df8a6a None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:59.136812 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 341/679] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:32:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:32:59.141890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88ed32-2a4f-4be8-ae74-0eea736f39fc None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:59.168917 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc78c29-ccfa-4313-b412-65778b9f38dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:59.169626 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc78c29-ccfa-4313-b412-65778b9f38dd None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:59.169889 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc78c29-ccfa-4313-b412-65778b9f38dd None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:59.170162 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc78c29-ccfa-4313-b412-65778b9f38dd None admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:59.171005 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc78c29-ccfa-4313-b412-65778b9f38dd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:59.171312 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc78c29-ccfa-4313-b412-65778b9f38dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:59.172723 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc78c29-ccfa-4313-b412-65778b9f38dd None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:59.197768 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 339/680] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Jul 6 13:32:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:59.199088 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc78c29-ccfa-4313-b412-65778b9f38dd None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:59.202676 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:59.251853 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88ed32-2a4f-4be8-ae74-0eea736f39fc None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:59.287048 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64702b-4ab3-4dbd-9b2f-940e403216ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:59.287714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64702b-4ab3-4dbd-9b2f-940e403216ff None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:59.287994 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64702b-4ab3-4dbd-9b2f-940e403216ff None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:59.288268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64702b-4ab3-4dbd-9b2f-940e403216ff None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:59.289022 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64702b-4ab3-4dbd-9b2f-940e403216ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:59.289330 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64702b-4ab3-4dbd-9b2f-940e403216ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:59.290696 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64702b-4ab3-4dbd-9b2f-940e403216ff None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:59.308929 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 342/681] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:32:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:59.310084 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64702b-4ab3-4dbd-9b2f-940e403216ff None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:59.311854 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc78c29-ccfa-4313-b412-65778b9f38dd None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:59.351817 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202885a-b904-44c8-afbe-9d690e66a04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:59.352488 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202885a-b904-44c8-afbe-9d690e66a04a None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:59.352777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202885a-b904-44c8-afbe-9d690e66a04a None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:59.353089 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202885a-b904-44c8-afbe-9d690e66a04a None admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:59.353890 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4202885a-b904-44c8-afbe-9d690e66a04a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:59.354201 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4202885a-b904-44c8-afbe-9d690e66a04a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:59.356217 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4202885a-b904-44c8-afbe-9d690e66a04a None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:59.361097 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:59.369368 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 340/682] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:32:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:59.370979 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202885a-b904-44c8-afbe-9d690e66a04a None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:59.436789 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:59.439150 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64702b-4ab3-4dbd-9b2f-940e403216ff None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:59.478008 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202885a-b904-44c8-afbe-9d690e66a04a None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:59.479992 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b459a5-9450-4314-bb25-611651516685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:59.481553 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b459a5-9450-4314-bb25-611651516685 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:59.481838 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b459a5-9450-4314-bb25-611651516685 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:59.482113 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b459a5-9450-4314-bb25-611651516685 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:59.493233 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b459a5-9450-4314-bb25-611651516685 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:32:59.501375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-17b459a5-9450-4314-bb25-611651516685 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:32:59.501375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:59.501375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:59.501375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:59.501375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:59.501375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:59.501375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:59.501375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:59.501375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:59.501375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:59.501375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:59.501375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:59.501375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:32:59.501375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:32:59.501375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:32:59.501375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:32:59.502409 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:32:59.502409 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:32:59.502409 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:32:59.502409 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:32:59.502409 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:59.502409 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:59.502409 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:59.502409 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:59.502409 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:59.502409 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:59.502409 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:59.502409 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:59.502409 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:59.502409 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:59.502409 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:59.502409 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:59.503239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:59.503239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:59.503239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:59.503239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:59.503239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:32:59.503239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:32:59.503239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:32:59.503239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:32:59.503239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:32:59.503239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:32:59.503239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:32:59.503239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:59.503239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 343/683] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:32:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:59.504569 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b459a5-9450-4314-bb25-611651516685 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:59.518883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18ed126-c72b-49b7-91f2-07918836ea08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:59.519858 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ed126-c72b-49b7-91f2-07918836ea08 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:59.520150 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ed126-c72b-49b7-91f2-07918836ea08 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:59.520737 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ed126-c72b-49b7-91f2-07918836ea08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:59.592901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18ed126-c72b-49b7-91f2-07918836ea08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:59.594386 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18ed126-c72b-49b7-91f2-07918836ea08 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:59.611421 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:59.615896 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b459a5-9450-4314-bb25-611651516685 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:59.653427 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6e2ba-8265-4c83-b786-e8bde8f9d938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:59.654147 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6e2ba-8265-4c83-b786-e8bde8f9d938 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:59.654442 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6e2ba-8265-4c83-b786-e8bde8f9d938 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:59.654720 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6e2ba-8265-4c83-b786-e8bde8f9d938 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:59.657693 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:59.662096 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f6e2ba-8265-4c83-b786-e8bde8f9d938 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:59.663761 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f6e2ba-8265-4c83-b786-e8bde8f9d938 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:59.682919 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-54f6e2ba-8265-4c83-b786-e8bde8f9d938 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:32:59.682919 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:32:59.682919 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:32:59.682919 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:32:59.682919 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:32:59.682919 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:32:59.682919 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:32:59.682919 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:32:59.682919 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:32:59.682919 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:32:59.682919 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:32:59.682919 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:32:59.682919 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:32:59.682919 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:32:59.682919 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:32:59.684044 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:32:59.684044 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:32:59.684044 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:32:59.684044 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:32:59.684044 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:32:59.684044 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:32:59.684044 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:32:59.684044 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:32:59.684044 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:32:59.684044 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:32:59.684044 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:32:59.684044 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:32:59.684044 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:32:59.684044 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:32:59.684044 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:32:59.684044 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:32:59.684044 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:32:59.686088 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:32:59.686088 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:32:59.686088 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:32:59.686088 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:32:59.686088 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:32:59.686088 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:32:59.686088 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:32:59.686088 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:32:59.686088 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 344/684] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:32:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:32:59.686088 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6e2ba-8265-4c83-b786-e8bde8f9d938 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:59.717416 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 341/685] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:32:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:32:59.719221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18ed126-c72b-49b7-91f2-07918836ea08 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:59.798117 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6e2ba-8265-4c83-b786-e8bde8f9d938 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:59.824668 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18ed126-c72b-49b7-91f2-07918836ea08 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:32:59.837971 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b759d-6c35-4ac8-bc99-12309fbb5294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:59.839326 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b759d-6c35-4ac8-bc99-12309fbb5294 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:59.839798 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b759d-6c35-4ac8-bc99-12309fbb5294 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:59.840187 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b759d-6c35-4ac8-bc99-12309fbb5294 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:59.840913 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b759d-6c35-4ac8-bc99-12309fbb5294 None admin] RBAC: Adding query filter params (role.id=a6853d485f7942259b3646de3f386852, scope.project.id=bfe6c3d8da2743be9609184db2506303, user.id=5997058ea0fe456a89856e222722a5cc) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:59.841251 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b759d-6c35-4ac8-bc99-12309fbb5294 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:59.842664 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b759d-6c35-4ac8-bc99-12309fbb5294 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:59.860685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5336c90a-5aaf-49c2-adf8-f77c2913821e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:32:59.861340 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 345/686] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:32:59 2022] GET /identity/v3/role_assignments?user.id=5997058ea0fe456a89856e222722a5cc&scope.project.id=bfe6c3d8da2743be9609184db2506303&role.id=a6853d485f7942259b3646de3f386852 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:59.861452 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336c90a-5aaf-49c2-adf8-f77c2913821e None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:32:59.861684 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336c90a-5aaf-49c2-adf8-f77c2913821e None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:32:59.861964 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336c90a-5aaf-49c2-adf8-f77c2913821e None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:32:59.862182 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b759d-6c35-4ac8-bc99-12309fbb5294 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:59.862729 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5336c90a-5aaf-49c2-adf8-f77c2913821e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:32:59.863046 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5336c90a-5aaf-49c2-adf8-f77c2913821e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:32:59.864468 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5336c90a-5aaf-49c2-adf8-f77c2913821e None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:32:59.880614 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 342/687] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:32:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:32:59.880614 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5336c90a-5aaf-49c2-adf8-f77c2913821e None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:32:59.900515 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:59.926470 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:32:59.991146 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b759d-6c35-4ac8-bc99-12309fbb5294 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:00.037747 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5336c90a-5aaf-49c2-adf8-f77c2913821e None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:00.044087 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a86036-ec08-4609-ad79-c7e2551fb4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:00.044756 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a86036-ec08-4609-ad79-c7e2551fb4f0 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:00.045065 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a86036-ec08-4609-ad79-c7e2551fb4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:00.045342 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a86036-ec08-4609-ad79-c7e2551fb4f0 None admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:00.046143 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a86036-ec08-4609-ad79-c7e2551fb4f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:00.046449 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a86036-ec08-4609-ad79-c7e2551fb4f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:00.047881 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a86036-ec08-4609-ad79-c7e2551fb4f0 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:00.063396 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 346/688] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:32:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:00.064102 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a86036-ec08-4609-ad79-c7e2551fb4f0 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:00.083493 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d75be-69e8-4f44-a3b2-c3fefa0d4b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:00.084289 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d75be-69e8-4f44-a3b2-c3fefa0d4b34 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:00.084572 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d75be-69e8-4f44-a3b2-c3fefa0d4b34 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:00.084850 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d75be-69e8-4f44-a3b2-c3fefa0d4b34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:00.098207 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d75be-69e8-4f44-a3b2-c3fefa0d4b34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:00.100229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d75be-69e8-4f44-a3b2-c3fefa0d4b34 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:00.123807 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-bf3d75be-69e8-4f44-a3b2-c3fefa0d4b34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:00.123807 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:00.123807 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:00.123807 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:00.123807 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:00.123807 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:00.123807 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:00.123807 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:00.123807 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:00.123807 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:00.123807 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:00.123807 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:00.123807 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:00.123807 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:00.123807 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:00.124777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:00.124777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:00.124777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:00.124777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:00.124777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:00.124777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:00.124777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:00.124777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:00.124777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:00.124777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:00.124777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:00.124777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:00.124777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:00.124777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:00.124777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:00.124777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:00.124777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:00.125777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:00.125777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:00.125777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:00.125777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:00.126740 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 343/689] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:32:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:00.127268 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d75be-69e8-4f44-a3b2-c3fefa0d4b34 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:00.171034 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:00.194069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a86036-ec08-4609-ad79-c7e2551fb4f0 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:00.232112 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16270301-ce18-4b2d-a149-0ee94a0fe343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:00.233057 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16270301-ce18-4b2d-a149-0ee94a0fe343 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:00.233409 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16270301-ce18-4b2d-a149-0ee94a0fe343 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:00.233756 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16270301-ce18-4b2d-a149-0ee94a0fe343 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:00.245020 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16270301-ce18-4b2d-a149-0ee94a0fe343 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:00.246569 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16270301-ce18-4b2d-a149-0ee94a0fe343 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:00.249364 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:00.264536 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-16270301-ce18-4b2d-a149-0ee94a0fe343 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:00.264536 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:00.264536 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:00.264536 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:00.264536 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:00.264536 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:00.264536 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:00.264536 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:00.264536 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:00.264536 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:00.264536 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:00.264536 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:00.264536 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:00.264536 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:00.264536 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:00.265674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:00.265674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:00.265674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:00.265674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:00.265674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:00.265674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:00.265674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:00.265674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:00.265674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:00.265674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:00.265674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:00.265674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:00.265674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:00.265674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:00.265674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:00.265674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:00.265674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:00.266798 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:00.266798 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:00.266798 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:00.266798 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:00.268907 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d75be-69e8-4f44-a3b2-c3fefa0d4b34 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:00.272767 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 347/690] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:00.273738 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16270301-ce18-4b2d-a149-0ee94a0fe343 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:00.326159 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbf817-9b2d-4beb-8d17-7c7fd5c4a7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:00.327260 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbf817-9b2d-4beb-8d17-7c7fd5c4a7b3 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:00.327697 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbf817-9b2d-4beb-8d17-7c7fd5c4a7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:00.328127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbf817-9b2d-4beb-8d17-7c7fd5c4a7b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:00.329449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbbf817-9b2d-4beb-8d17-7c7fd5c4a7b3 None admin] RBAC: Adding query filter params (role.id=a6853d485f7942259b3646de3f386852, scope.project.id=bfe6c3d8da2743be9609184db2506303, user.id=b9f250f5dbc54d2e9293962ece848f6c) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:00.329961 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbbf817-9b2d-4beb-8d17-7c7fd5c4a7b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:00.332574 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbbf817-9b2d-4beb-8d17-7c7fd5c4a7b3 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:00.343669 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 344/691] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:33:00 2022] GET /identity/v3/role_assignments?user.id=b9f250f5dbc54d2e9293962ece848f6c&scope.project.id=bfe6c3d8da2743be9609184db2506303&role.id=a6853d485f7942259b3646de3f386852 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:00.345461 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbf817-9b2d-4beb-8d17-7c7fd5c4a7b3 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:00.430172 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16270301-ce18-4b2d-a149-0ee94a0fe343 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:00.460490 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbf817-9b2d-4beb-8d17-7c7fd5c4a7b3 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:00.473412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f10cf-4b1a-4972-9c06-6a51e5092f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:00.474462 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f10cf-4b1a-4972-9c06-6a51e5092f79 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:00.474928 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f10cf-4b1a-4972-9c06-6a51e5092f79 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:00.475360 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f10cf-4b1a-4972-9c06-6a51e5092f79 None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:00.476254 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f10cf-4b1a-4972-9c06-6a51e5092f79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:00.476725 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f10cf-4b1a-4972-9c06-6a51e5092f79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:00.480300 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f10cf-4b1a-4972-9c06-6a51e5092f79 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:00.482450 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:00.495871 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 348/692] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:00.500795 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f10cf-4b1a-4972-9c06-6a51e5092f79 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:00.515250 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd63d8-1e19-43d7-b64f-ccbf4687625c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:00.516280 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd63d8-1e19-43d7-b64f-ccbf4687625c None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:00.516445 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd63d8-1e19-43d7-b64f-ccbf4687625c None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:00.516827 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd63d8-1e19-43d7-b64f-ccbf4687625c None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:00.517719 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd63d8-1e19-43d7-b64f-ccbf4687625c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:00.518101 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd63d8-1e19-43d7-b64f-ccbf4687625c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:00.519571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd63d8-1e19-43d7-b64f-ccbf4687625c None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:00.547065 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 345/693] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:00.552917 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd63d8-1e19-43d7-b64f-ccbf4687625c None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:00.558910 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:00.623969 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f10cf-4b1a-4972-9c06-6a51e5092f79 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:00.673138 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46535cb0-0d91-463f-a788-8d9df8cbc1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:00.674895 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46535cb0-0d91-463f-a788-8d9df8cbc1f3 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:00.675373 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46535cb0-0d91-463f-a788-8d9df8cbc1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:00.675792 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46535cb0-0d91-463f-a788-8d9df8cbc1f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:00.685985 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46535cb0-0d91-463f-a788-8d9df8cbc1f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:00.688012 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46535cb0-0d91-463f-a788-8d9df8cbc1f3 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:00.706825 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-46535cb0-0d91-463f-a788-8d9df8cbc1f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:00.706825 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:00.706825 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:00.706825 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:00.706825 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:00.706825 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:00.706825 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:00.706825 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:00.706825 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:00.706825 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:00.706825 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:00.706825 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:00.706825 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:33:00.706825 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:33:00.706825 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:33:00.707938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:33:00.707938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:00.707938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:00.707938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:00.707938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:00.707938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:00.707938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:00.707938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:00.707938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:00.707938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:00.707938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:00.707938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:00.707938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:00.707938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:00.707938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:00.707938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:00.707938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:00.709045 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:33:00.709045 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:33:00.709045 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:33:00.709045 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:33:00.709045 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:33:00.709045 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:33:00.709045 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:00.709045 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:00.711119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 349/694] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:33:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:00.717825 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46535cb0-0d91-463f-a788-8d9df8cbc1f3 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:00.725033 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd63d8-1e19-43d7-b64f-ccbf4687625c None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:00.765181 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507cbef5-a2da-4385-8c30-0ea4c101f217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:00.766195 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507cbef5-a2da-4385-8c30-0ea4c101f217 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:00.766594 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507cbef5-a2da-4385-8c30-0ea4c101f217 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:00.766971 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507cbef5-a2da-4385-8c30-0ea4c101f217 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:00.776227 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507cbef5-a2da-4385-8c30-0ea4c101f217 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:00.778015 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507cbef5-a2da-4385-8c30-0ea4c101f217 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:00.790831 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:00.795435 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-507cbef5-a2da-4385-8c30-0ea4c101f217 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 06 13:33:00.795435 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:00.795435 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:00.795435 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:00.795435 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:00.795435 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:00.795435 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:00.795435 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:00.795435 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:00.795435 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:00.795435 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:00.795435 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:00.795435 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:33:00.795435 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:33:00.795435 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:33:00.795435 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:33:00.796542 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:00.796542 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:00.796542 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:33:00.796542 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:00.796542 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:33:00.796542 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:00.796542 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:00.796542 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:00.796542 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:00.796542 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:00.796542 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:00.796542 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:00.796542 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:00.796542 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:00.796542 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:00.796542 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:00.796542 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:00.796542 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:00.797738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:00.797738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:00.797738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:33:00.797738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:33:00.797738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:33:00.797738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:33:00.797738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:33:00.797738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:33:00.797738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 06 13:33:00.797738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:00.800769 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 346/695] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Jul 6 13:33:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:33:00.806219 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507cbef5-a2da-4385-8c30-0ea4c101f217 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:00.849405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46535cb0-0d91-463f-a788-8d9df8cbc1f3 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:00.877595 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:00.890786 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6587c20-e441-4901-9299-907480f182f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:00.892435 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6587c20-e441-4901-9299-907480f182f0 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:00.892881 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6587c20-e441-4901-9299-907480f182f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:00.893321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6587c20-e441-4901-9299-907480f182f0 None admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:00.894261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6587c20-e441-4901-9299-907480f182f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:00.894703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6587c20-e441-4901-9299-907480f182f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:00.896292 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6587c20-e441-4901-9299-907480f182f0 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:00.918971 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 350/696] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:33:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:00.925020 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6587c20-e441-4901-9299-907480f182f0 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:00.943250 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507cbef5-a2da-4385-8c30-0ea4c101f217 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:01.010520 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176c40e-d3f0-4c53-a6ec-7e2ec79001ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:01.011211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176c40e-d3f0-4c53-a6ec-7e2ec79001ef None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:01.011498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176c40e-d3f0-4c53-a6ec-7e2ec79001ef None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:01.011892 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176c40e-d3f0-4c53-a6ec-7e2ec79001ef None admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:01.012873 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176c40e-d3f0-4c53-a6ec-7e2ec79001ef None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:01.013218 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176c40e-d3f0-4c53-a6ec-7e2ec79001ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:01.014675 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176c40e-d3f0-4c53-a6ec-7e2ec79001ef None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:01.035828 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 347/697] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Jul 6 13:33:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:01.041815 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12162ff-8855-4715-bee2-411697bc4f0d None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:01.042108 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12162ff-8855-4715-bee2-411697bc4f0d None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:01.042388 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12162ff-8855-4715-bee2-411697bc4f0d None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:01.043805 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 348/698] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:01.045250 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12162ff-8855-4715-bee2-411697bc4f0d None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:01.069874 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6587c20-e441-4901-9299-907480f182f0 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:01.086152 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:01.110412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df9b27a-0bbf-45a7-853f-80e577231daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:01.111254 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9b27a-0bbf-45a7-853f-80e577231daa None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:01.111592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9b27a-0bbf-45a7-853f-80e577231daa None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:01.112144 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9b27a-0bbf-45a7-853f-80e577231daa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:01.112801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df9b27a-0bbf-45a7-853f-80e577231daa None admin] RBAC: Adding query filter params (role.id=a6853d485f7942259b3646de3f386852, scope.project.id=bfe6c3d8da2743be9609184db2506303, user.id=8c1bea05c37646b0a05fa8dda0367ef2) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:01.113146 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df9b27a-0bbf-45a7-853f-80e577231daa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:01.114680 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df9b27a-0bbf-45a7-853f-80e577231daa None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:01.122307 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 351/699] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:33:00 2022] GET /identity/v3/role_assignments?user.id=8c1bea05c37646b0a05fa8dda0367ef2&scope.project.id=bfe6c3d8da2743be9609184db2506303&role.id=a6853d485f7942259b3646de3f386852 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:01.123733 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34eea9c-588d-42c2-8c36-e4714bf952cb None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:01.124025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34eea9c-588d-42c2-8c36-e4714bf952cb None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:01.124347 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34eea9c-588d-42c2-8c36-e4714bf952cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:01.143063 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-d34eea9c-588d-42c2-8c36-e4714bf952cb None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:01.144950 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-d34eea9c-588d-42c2-8c36-e4714bf952cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:01.190530 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12162ff-8855-4715-bee2-411697bc4f0d None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:01.219328 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:01.250072 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a9dc38-29b9-420d-b362-412213be4676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:01.251173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9dc38-29b9-420d-b362-412213be4676 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:01.251690 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9dc38-29b9-420d-b362-412213be4676 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:01.252218 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9dc38-29b9-420d-b362-412213be4676 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:01.308643 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 352/700] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:33:01.336595 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34eea9c-588d-42c2-8c36-e4714bf952cb None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:01.373480 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a9dc38-29b9-420d-b362-412213be4676 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:01.375704 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a9dc38-29b9-420d-b362-412213be4676 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:01.456674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34eea9c-588d-42c2-8c36-e4714bf952cb None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:01.467345 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:01.482121 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 349/701] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:33:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:33:01.488386 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a9dc38-29b9-420d-b362-412213be4676 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:01.503215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104df50e-80a1-458d-8c93-d226b57b06f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:01.504127 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104df50e-80a1-458d-8c93-d226b57b06f4 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:01.504407 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104df50e-80a1-458d-8c93-d226b57b06f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:01.504691 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104df50e-80a1-458d-8c93-d226b57b06f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:01.509976 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:01.519157 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104df50e-80a1-458d-8c93-d226b57b06f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:01.520970 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104df50e-80a1-458d-8c93-d226b57b06f4 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:01.542863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-104df50e-80a1-458d-8c93-d226b57b06f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:01.542863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:01.542863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:01.542863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:01.542863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:01.542863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:01.542863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:01.542863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:01.542863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:01.542863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:01.542863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:01.542863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:01.542863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:01.542863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:01.542863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:01.543826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:01.543826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:01.543826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:01.543826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:01.543826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:01.543826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:01.543826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:01.543826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:01.543826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:01.543826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:01.543826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:01.543826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:01.543826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:01.543826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:01.543826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:01.543826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:01.543826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:01.544725 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:01.544725 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:01.544725 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:01.544725 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:01.544725 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 353/702] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:33:01.559805 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104df50e-80a1-458d-8c93-d226b57b06f4 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:01.600621 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a9dc38-29b9-420d-b362-412213be4676 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:01.645025 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07616580-1dbd-477f-aff6-957e1454fcef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:01.645025 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07616580-1dbd-477f-aff6-957e1454fcef None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:01.645025 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07616580-1dbd-477f-aff6-957e1454fcef None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:01.645025 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07616580-1dbd-477f-aff6-957e1454fcef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:01.665761 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07616580-1dbd-477f-aff6-957e1454fcef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:01.668360 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07616580-1dbd-477f-aff6-957e1454fcef None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:01.708342 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104df50e-80a1-458d-8c93-d226b57b06f4 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:01.721756 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-07616580-1dbd-477f-aff6-957e1454fcef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:01.721756 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:01.721756 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:01.721756 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:01.721756 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:01.721756 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:01.721756 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:01.721756 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:01.721756 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:01.721756 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:01.721756 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:01.721756 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:01.721756 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:01.721756 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:01.721756 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:01.722709 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:01.722709 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:01.722709 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:01.722709 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:01.722709 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:01.722709 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:01.722709 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:01.722709 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:01.722709 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:01.722709 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:01.722709 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:01.722709 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:01.722709 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:01.722709 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:01.722709 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:01.722709 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:01.722709 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:01.723599 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:01.723599 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:01.723599 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:01.723599 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:01.727496 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 350/703] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:33:01.728681 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07616580-1dbd-477f-aff6-957e1454fcef None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:01.797730 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab45ff89-acdb-4cf1-a61d-2982a62e1695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:01.799108 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45ff89-acdb-4cf1-a61d-2982a62e1695 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:01.799620 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45ff89-acdb-4cf1-a61d-2982a62e1695 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:01.800098 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45ff89-acdb-4cf1-a61d-2982a62e1695 None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:01.801095 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab45ff89-acdb-4cf1-a61d-2982a62e1695 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:01.801603 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab45ff89-acdb-4cf1-a61d-2982a62e1695 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:01.803186 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab45ff89-acdb-4cf1-a61d-2982a62e1695 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:01.822378 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 354/704] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:01.825355 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:01.827291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab45ff89-acdb-4cf1-a61d-2982a62e1695 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:01.873698 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:01.874995 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07616580-1dbd-477f-aff6-957e1454fcef None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:01.933858 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c252cf1-2dfb-403e-adde-e92dfcf71ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:01.934533 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c252cf1-2dfb-403e-adde-e92dfcf71ca4 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:01.934814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c252cf1-2dfb-403e-adde-e92dfcf71ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:01.935087 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c252cf1-2dfb-403e-adde-e92dfcf71ca4 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:01.935821 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c252cf1-2dfb-403e-adde-e92dfcf71ca4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:01.936134 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c252cf1-2dfb-403e-adde-e92dfcf71ca4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:01.937532 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c252cf1-2dfb-403e-adde-e92dfcf71ca4 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:01.965042 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 351/705] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:01.970876 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c252cf1-2dfb-403e-adde-e92dfcf71ca4 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:02.037183 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab45ff89-acdb-4cf1-a61d-2982a62e1695 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:02.090456 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8016c15-9879-4a75-9959-c050c2f2d6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:02.091051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8016c15-9879-4a75-9959-c050c2f2d6f4 None admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:02.091330 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8016c15-9879-4a75-9959-c050c2f2d6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:02.091604 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8016c15-9879-4a75-9959-c050c2f2d6f4 None admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:02.092324 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8016c15-9879-4a75-9959-c050c2f2d6f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:02.093819 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8016c15-9879-4a75-9959-c050c2f2d6f4 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:02.133136 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 355/706] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jul 6 13:33:01 2022] POST /identity/v3/users => generated 254 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:33:02.157833 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c252cf1-2dfb-403e-adde-e92dfcf71ca4 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:02.193407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:02.198338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95e6cc-a6a6-4775-b2c4-e987b0fcc3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:02.199324 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95e6cc-a6a6-4775-b2c4-e987b0fcc3aa None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:02.199613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95e6cc-a6a6-4775-b2c4-e987b0fcc3aa None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:02.199893 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95e6cc-a6a6-4775-b2c4-e987b0fcc3aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:02.207474 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b95e6cc-a6a6-4775-b2c4-e987b0fcc3aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:02.208952 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b95e6cc-a6a6-4775-b2c4-e987b0fcc3aa None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:02.224612 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-7b95e6cc-a6a6-4775-b2c4-e987b0fcc3aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:02.224612 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:02.224612 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:02.224612 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:02.224612 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:02.224612 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:02.224612 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:02.224612 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:02.224612 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:02.224612 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:02.224612 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:02.224612 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:02.224612 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:33:02.224612 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:33:02.224612 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:33:02.225766 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:33:02.225766 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:02.225766 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:02.225766 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:02.225766 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:02.225766 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:02.225766 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:02.225766 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:02.225766 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:02.225766 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:02.225766 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:02.225766 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:02.225766 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:02.225766 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:02.225766 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:02.225766 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:02.225766 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:02.226890 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:33:02.226890 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:33:02.226890 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:33:02.226890 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:33:02.226890 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:33:02.226890 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:33:02.226890 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:02.226890 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:02.230289 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 352/707] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:33:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:02.238179 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8016c15-9879-4a75-9959-c050c2f2d6f4 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:02.280054 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147e705-4206-457f-aa7f-e80f1b410a0d None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:02.280335 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147e705-4206-457f-aa7f-e80f1b410a0d None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:02.280612 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147e705-4206-457f-aa7f-e80f1b410a0d None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:02.282116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 353/708] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:33:02.287671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180391d-c6d7-4f64-9b48-64e773b18ded None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:02.287967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180391d-c6d7-4f64-9b48-64e773b18ded None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:02.288251 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180391d-c6d7-4f64-9b48-64e773b18ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:02.325221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-3180391d-c6d7-4f64-9b48-64e773b18ded None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:02.326459 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-3180391d-c6d7-4f64-9b48-64e773b18ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:02.392830 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:02.426221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8016c15-9879-4a75-9959-c050c2f2d6f4 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:02.445475 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 354/709] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:33:02.467840 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180391d-c6d7-4f64-9b48-64e773b18ded None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:02.478097 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb8a0b0-ec12-4ee5-9bd6-61541e545765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:02.479126 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb8a0b0-ec12-4ee5-9bd6-61541e545765 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:02.479571 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb8a0b0-ec12-4ee5-9bd6-61541e545765 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:02.480261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb8a0b0-ec12-4ee5-9bd6-61541e545765 None admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:02.481295 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb8a0b0-ec12-4ee5-9bd6-61541e545765 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:02.481750 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb8a0b0-ec12-4ee5-9bd6-61541e545765 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:02.484277 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb8a0b0-ec12-4ee5-9bd6-61541e545765 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:02.498488 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 356/710] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:33:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:02.504456 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb8a0b0-ec12-4ee5-9bd6-61541e545765 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:02.601968 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180391d-c6d7-4f64-9b48-64e773b18ded None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:02.629637 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:02.647197 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf66bdb-7d36-4172-a9a8-a52406e83e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:02.649887 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb8a0b0-ec12-4ee5-9bd6-61541e545765 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:02.650412 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf66bdb-7d36-4172-a9a8-a52406e83e3a None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:02.651379 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf66bdb-7d36-4172-a9a8-a52406e83e3a None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:02.651804 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf66bdb-7d36-4172-a9a8-a52406e83e3a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:02.653722 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf66bdb-7d36-4172-a9a8-a52406e83e3a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:02.655292 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf66bdb-7d36-4172-a9a8-a52406e83e3a None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:02.683078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-adf66bdb-7d36-4172-a9a8-a52406e83e3a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 06 13:33:02.683078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:02.683078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 13:33:02.683078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 13:33:02.683078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 13:33:02.683078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 13:33:02.683078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 13:33:02.683078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:02.683078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 06 13:33:02.683078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:02.683078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 13:33:02.683078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:02.683078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:02.683078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:02.683078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:02.683078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:02.684349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:02.684349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:02.684349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:02.684349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:02.684349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:02.684349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:02.684349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:02.684349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 13:33:02.684349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 13:33:02.684349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 13:33:02.684349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 13:33:02.684349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:02.684349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:02.684349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:02.684349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:02.684349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:02.685484 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:02.685484 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:02.685484 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:02.685484 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:02.685484 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:02.685484 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:02.685484 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:02.685484 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:02.685484 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:02.685484 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:02.685484 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:02.685484 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 13:33:02.685484 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:02.685484 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 06 13:33:02.685484 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:02.689648 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 355/711] 10.222.0.20 () {56 vars in 1129 bytes} [Wed Jul 6 13:33:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:33:02.695845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf66bdb-7d36-4172-a9a8-a52406e83e3a None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:02.724053 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb6d01-e632-4b77-8a5b-2a4c5c6ea6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:02.725077 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb6d01-e632-4b77-8a5b-2a4c5c6ea6ec None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:02.727299 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb6d01-e632-4b77-8a5b-2a4c5c6ea6ec None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:02.727771 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb6d01-e632-4b77-8a5b-2a4c5c6ea6ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:02.728878 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeb6d01-e632-4b77-8a5b-2a4c5c6ea6ec None admin] RBAC: Adding query filter params (role.id=795daa2c78354fd48e4a741e1182cc98, scope.project.id=bfe6c3d8da2743be9609184db2506303, user.id=6647eb1e8a1544ef9d3a9806f2d1fc51) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:02.729657 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeb6d01-e632-4b77-8a5b-2a4c5c6ea6ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:02.732016 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeb6d01-e632-4b77-8a5b-2a4c5c6ea6ec None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:02.741842 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:02.746862 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 357/712] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:33:02 2022] GET /identity/v3/role_assignments?user.id=6647eb1e8a1544ef9d3a9806f2d1fc51&scope.project.id=bfe6c3d8da2743be9609184db2506303&role.id=795daa2c78354fd48e4a741e1182cc98 => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:02.838107 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf66bdb-7d36-4172-a9a8-a52406e83e3a None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:02.880532 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75220a94-13ca-4a21-867f-f1929573b623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:02.881236 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75220a94-13ca-4a21-867f-f1929573b623 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:02.881516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75220a94-13ca-4a21-867f-f1929573b623 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:02.881794 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75220a94-13ca-4a21-867f-f1929573b623 None admin] PATH_INFO: `/v3/services` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:02.882510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75220a94-13ca-4a21-867f-f1929573b623 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:02.882815 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75220a94-13ca-4a21-867f-f1929573b623 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:02.884196 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75220a94-13ca-4a21-867f-f1929573b623 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:02.890114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 356/713] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Jul 6 13:33:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:02.895487 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb6d01-e632-4b77-8a5b-2a4c5c6ea6ec None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:03.033059 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb6d01-e632-4b77-8a5b-2a4c5c6ea6ec None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:03.075918 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a23b5-34d6-47b2-b5a5-0326207673ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:03.076592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a23b5-34d6-47b2-b5a5-0326207673ef None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:03.076871 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a23b5-34d6-47b2-b5a5-0326207673ef None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:03.077206 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a23b5-34d6-47b2-b5a5-0326207673ef None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:03.077914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a23b5-34d6-47b2-b5a5-0326207673ef None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:03.078220 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a23b5-34d6-47b2-b5a5-0326207673ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:03.079569 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a23b5-34d6-47b2-b5a5-0326207673ef None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:03.086899 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 358/714] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Jul 6 13:33:02 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:03.115803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320e060-d2a2-4759-8a8d-aa892d98e1eb None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:03.116455 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320e060-d2a2-4759-8a8d-aa892d98e1eb None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:03.116965 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320e060-d2a2-4759-8a8d-aa892d98e1eb None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:03.118966 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 357/715] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:03.125397 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa63d5-e89e-4da2-97bc-13b75a7d932b None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:03.126005 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa63d5-e89e-4da2-97bc-13b75a7d932b None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:03.126542 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa63d5-e89e-4da2-97bc-13b75a7d932b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:03.150661 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-c2aa63d5-e89e-4da2-97bc-13b75a7d932b None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:03.152272 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-c2aa63d5-e89e-4da2-97bc-13b75a7d932b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:03.234714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:03.249164 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 359/716] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:33:03.281845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320e060-d2a2-4759-8a8d-aa892d98e1eb None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:03.351201 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:03.412539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320e060-d2a2-4759-8a8d-aa892d98e1eb None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:03.456476 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f9bf7e-3b29-4f3a-9cc3-a809cdab5fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:03.457303 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9bf7e-3b29-4f3a-9cc3-a809cdab5fbe None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:03.457588 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9bf7e-3b29-4f3a-9cc3-a809cdab5fbe None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:03.457904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9bf7e-3b29-4f3a-9cc3-a809cdab5fbe None admin] PATH_INFO: `/v3/services/network` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:03.458562 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f9bf7e-3b29-4f3a-9cc3-a809cdab5fbe None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:03.459923 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f9bf7e-3b29-4f3a-9cc3-a809cdab5fbe None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:03.473361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-f3f9bf7e-3b29-4f3a-9cc3-a809cdab5fbe None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 06 13:33:03.473361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:03.473361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 13:33:03.473361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 13:33:03.473361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 13:33:03.473361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 13:33:03.473361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 13:33:03.473361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:03.473361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 06 13:33:03.473361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:03.473361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 13:33:03.473361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:03.473361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:03.473361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:03.473361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:03.473361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:03.473361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:03.474409 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:03.474409 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:03.474409 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:03.474409 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:03.474409 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:03.474409 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:03.474409 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 13:33:03.474409 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 13:33:03.474409 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 13:33:03.474409 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 13:33:03.474409 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:03.474409 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:03.474409 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:03.474409 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:03.474409 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:03.474409 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:03.476206 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:03.476206 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:03.476206 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:03.476206 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:03.476206 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:03.476206 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:03.476206 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:03.476206 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:03.476206 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:03.476206 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:03.476206 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 13:33:03.476206 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:03.476206 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 06 13:33:03.476206 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:03.476206 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 358/717] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:33:03 2022] GET /identity/v3/services/network => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:03.481965 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa63d5-e89e-4da2-97bc-13b75a7d932b None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:03.631276 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa63d5-e89e-4da2-97bc-13b75a7d932b None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:03.677411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10640b1e-9ebd-4bf8-89e3-ef058b48b93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:03.678453 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10640b1e-9ebd-4bf8-89e3-ef058b48b93e None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:03.678965 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10640b1e-9ebd-4bf8-89e3-ef058b48b93e None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:03.679444 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10640b1e-9ebd-4bf8-89e3-ef058b48b93e None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:03.680356 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10640b1e-9ebd-4bf8-89e3-ef058b48b93e None admin] RBAC: Adding query filter params (name=network) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:03.680882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10640b1e-9ebd-4bf8-89e3-ef058b48b93e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:03.683049 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10640b1e-9ebd-4bf8-89e3-ef058b48b93e None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:03.688588 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 360/718] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Jul 6 13:33:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:03.698276 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f9bf7e-3b29-4f3a-9cc3-a809cdab5fbe None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:03.829258 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9b888-0406-4be4-b111-9ebda75fff3d None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:03.829613 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9b888-0406-4be4-b111-9ebda75fff3d None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:03.829824 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9b888-0406-4be4-b111-9ebda75fff3d None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:03.831494 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 361/719] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:03.836378 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd153f80-0bf3-4512-9ed2-4b12970f6c6e None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:03.836665 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd153f80-0bf3-4512-9ed2-4b12970f6c6e None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:03.836947 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd153f80-0bf3-4512-9ed2-4b12970f6c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:03.848733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f9bf7e-3b29-4f3a-9cc3-a809cdab5fbe None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:03.854437 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-fd153f80-0bf3-4512-9ed2-4b12970f6c6e None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:03.855778 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-fd153f80-0bf3-4512-9ed2-4b12970f6c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:03.879349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:03.911942 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655cd85-9bba-40b8-8986-aa53677a8e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:03.912539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655cd85-9bba-40b8-8986-aa53677a8e1b None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:03.912824 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655cd85-9bba-40b8-8986-aa53677a8e1b None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:03.913129 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655cd85-9bba-40b8-8986-aa53677a8e1b None admin] PATH_INFO: `/v3/services` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:03.913313 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:03.913855 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b655cd85-9bba-40b8-8986-aa53677a8e1b None admin] RBAC: Adding query filter params (type=network) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:03.914166 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b655cd85-9bba-40b8-8986-aa53677a8e1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:03.915582 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b655cd85-9bba-40b8-8986-aa53677a8e1b None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:03.928276 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 359/720] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Jul 6 13:33:03 2022] GET /identity/v3/services?type=network => generated 124 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:03.941667 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 362/721] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:33:03.966649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655cd85-9bba-40b8-8986-aa53677a8e1b None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:04.088128 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655cd85-9bba-40b8-8986-aa53677a8e1b None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:04.141751 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b4844c-8d5f-4be8-a8b6-a527ba3356f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:04.142573 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4844c-8d5f-4be8-a8b6-a527ba3356f2 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:04.142856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4844c-8d5f-4be8-a8b6-a527ba3356f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:04.143130 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4844c-8d5f-4be8-a8b6-a527ba3356f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:04.155024 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b4844c-8d5f-4be8-a8b6-a527ba3356f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:33:04.162883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-70b4844c-8d5f-4be8-a8b6-a527ba3356f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:33:04.162883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:04.162883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:04.162883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:04.162883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:04.162883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:04.162883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:04.162883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:04.162883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:04.162883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:04.162883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:04.162883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:04.162883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:33:04.162883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:33:04.162883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:33:04.162883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:33:04.163808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:33:04.163808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:33:04.163808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:33:04.163808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:33:04.163808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:04.163808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:04.163808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:04.163808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:04.163808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:04.163808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:04.163808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:04.163808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:04.163808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:04.163808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:04.163808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:04.163808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:04.164654 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:04.164654 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:04.164654 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:04.164654 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:04.164654 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:33:04.164654 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:33:04.164654 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:33:04.164654 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:33:04.164654 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:33:04.164654 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:33:04.164654 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:33:04.164654 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:04.164654 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 360/722] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:33:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:04.171240 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd153f80-0bf3-4512-9ed2-4b12970f6c6e None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:04.279749 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd153f80-0bf3-4512-9ed2-4b12970f6c6e None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:04.316586 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204afb6a-174b-4439-82fb-dcb110745c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:04.317284 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204afb6a-174b-4439-82fb-dcb110745c64 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:04.317564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204afb6a-174b-4439-82fb-dcb110745c64 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:04.317837 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204afb6a-174b-4439-82fb-dcb110745c64 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:04.318581 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204afb6a-174b-4439-82fb-dcb110745c64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:04.318884 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204afb6a-174b-4439-82fb-dcb110745c64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:04.320238 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204afb6a-174b-4439-82fb-dcb110745c64 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:04.329235 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 363/723] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:33:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:04.334709 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b4844c-8d5f-4be8-a8b6-a527ba3356f2 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:04.382610 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:04.454959 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b4844c-8d5f-4be8-a8b6-a527ba3356f2 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:04.489174 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d27367-abef-4f9a-954d-93e91d87fe98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:04.490020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d27367-abef-4f9a-954d-93e91d87fe98 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:04.490305 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d27367-abef-4f9a-954d-93e91d87fe98 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:04.490577 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d27367-abef-4f9a-954d-93e91d87fe98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:04.501649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d27367-abef-4f9a-954d-93e91d87fe98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:04.503368 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d27367-abef-4f9a-954d-93e91d87fe98 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:04.532461 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-61d27367-abef-4f9a-954d-93e91d87fe98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:04.532461 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:04.532461 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:04.532461 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:04.532461 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:04.532461 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:04.532461 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:04.532461 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:04.532461 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:04.532461 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:04.532461 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:04.532461 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:04.532461 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:04.532461 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:04.532461 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:04.533406 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:04.533406 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:04.533406 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:04.533406 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:04.533406 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:04.533406 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:04.533406 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:04.533406 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:04.533406 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:04.533406 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:04.533406 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:04.533406 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:04.533406 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:04.533406 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:04.533406 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:04.533406 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:04.533406 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:04.534374 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:04.534374 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:04.534374 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:04.534374 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:04.536240 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 361/724] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:04.542298 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204afb6a-174b-4439-82fb-dcb110745c64 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:04.595127 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:04.649034 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204afb6a-174b-4439-82fb-dcb110745c64 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:04.686452 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7da17-5f17-4dc4-a27b-410eb91c3cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:04.687114 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7da17-5f17-4dc4-a27b-410eb91c3cc4 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:04.687394 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7da17-5f17-4dc4-a27b-410eb91c3cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:04.687673 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7da17-5f17-4dc4-a27b-410eb91c3cc4 None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:04.688390 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7da17-5f17-4dc4-a27b-410eb91c3cc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:04.688695 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7da17-5f17-4dc4-a27b-410eb91c3cc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:04.690082 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7da17-5f17-4dc4-a27b-410eb91c3cc4 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:04.702719 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 364/725] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:04.708230 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d27367-abef-4f9a-954d-93e91d87fe98 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:04.810738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:04.824461 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d27367-abef-4f9a-954d-93e91d87fe98 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:04.861971 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373045e-a25b-4404-8495-ba0f9c82a187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:04.862759 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373045e-a25b-4404-8495-ba0f9c82a187 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:04.863040 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373045e-a25b-4404-8495-ba0f9c82a187 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:04.863313 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373045e-a25b-4404-8495-ba0f9c82a187 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:04.878079 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a373045e-a25b-4404-8495-ba0f9c82a187 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:04.879590 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a373045e-a25b-4404-8495-ba0f9c82a187 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:04.896114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-a373045e-a25b-4404-8495-ba0f9c82a187 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 06 13:33:04.896114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:04.896114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:04.896114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:04.896114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:04.896114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:04.896114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:04.896114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:04.896114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:04.896114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:04.896114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:04.896114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:04.896114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:33:04.896114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:33:04.896114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:33:04.896114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:33:04.897117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:04.897117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:04.897117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:33:04.897117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:04.897117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:33:04.897117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:04.897117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:04.897117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:04.897117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:04.897117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:04.897117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:04.897117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:04.897117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:04.897117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:04.897117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:04.897117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:04.897117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:04.897117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:04.898058 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:04.898058 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:04.898058 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:33:04.898058 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:33:04.898058 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:33:04.898058 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:33:04.898058 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:33:04.898058 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:33:04.898058 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 06 13:33:04.898058 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:04.898058 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 362/726] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:33:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:04.903511 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7da17-5f17-4dc4-a27b-410eb91c3cc4 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:05.018434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7da17-5f17-4dc4-a27b-410eb91c3cc4 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:05.033362 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:05.054803 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e0d1d-9920-4388-849e-2779a349d158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:05.055477 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e0d1d-9920-4388-849e-2779a349d158 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:05.056167 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e0d1d-9920-4388-849e-2779a349d158 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:05.056464 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e0d1d-9920-4388-849e-2779a349d158 None admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:05.058254 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e0d1d-9920-4388-849e-2779a349d158 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:05.059035 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e0d1d-9920-4388-849e-2779a349d158 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:05.060472 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e0d1d-9920-4388-849e-2779a349d158 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:05.078451 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 365/727] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Jul 6 13:33:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:05.083958 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373045e-a25b-4404-8495-ba0f9c82a187 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:05.114635 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a988c-3a52-48a9-9762-ca1b7df62aec None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:05.114985 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a988c-3a52-48a9-9762-ca1b7df62aec None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:05.115196 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a988c-3a52-48a9-9762-ca1b7df62aec None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:05.117871 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 366/728] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:05.130046 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ff2fc-68b5-42b6-a331-bb5fbf37f3e1 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:05.130333 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ff2fc-68b5-42b6-a331-bb5fbf37f3e1 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:05.130610 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ff2fc-68b5-42b6-a331-bb5fbf37f3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:05.157740 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-d23ff2fc-68b5-42b6-a331-bb5fbf37f3e1 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:05.158930 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-d23ff2fc-68b5-42b6-a331-bb5fbf37f3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:05.269030 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 367/729] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 13:33:05.270776 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373045e-a25b-4404-8495-ba0f9c82a187 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:05.296588 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ff2fc-68b5-42b6-a331-bb5fbf37f3e1 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:05.351658 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a946cac-e5b9-4e16-b67e-91477ce3d057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:05.352319 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a946cac-e5b9-4e16-b67e-91477ce3d057 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:05.352603 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a946cac-e5b9-4e16-b67e-91477ce3d057 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:05.352877 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a946cac-e5b9-4e16-b67e-91477ce3d057 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:05.377610 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:05.469479 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a946cac-e5b9-4e16-b67e-91477ce3d057 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:05.470933 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a946cac-e5b9-4e16-b67e-91477ce3d057 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:05.477271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ff2fc-68b5-42b6-a331-bb5fbf37f3e1 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:05.481298 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:05.522189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06362d78-1df8-405a-a397-5b0181c52b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:05.522905 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06362d78-1df8-405a-a397-5b0181c52b97 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:05.523181 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06362d78-1df8-405a-a397-5b0181c52b97 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:05.523452 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06362d78-1df8-405a-a397-5b0181c52b97 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:05.542147 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06362d78-1df8-405a-a397-5b0181c52b97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:33:05.554051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-06362d78-1df8-405a-a397-5b0181c52b97 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:33:05.554051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:05.554051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:05.554051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:05.554051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:05.554051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:05.554051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:05.554051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:05.554051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:05.554051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:05.554051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:05.554051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:05.554051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:33:05.554051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:33:05.554051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:33:05.554051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:33:05.555493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:33:05.555493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:33:05.555493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:33:05.555493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:33:05.555493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:05.555493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:05.555493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:05.555493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:05.555493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:05.555493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:05.555493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:05.555493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:05.555493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:05.555493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:05.555493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:05.555493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:05.556328 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:05.556328 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:05.556328 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:05.556328 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:05.556328 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:33:05.556328 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:33:05.556328 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:33:05.556328 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:33:05.556328 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:33:05.556328 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:33:05.556328 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:33:05.556328 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:05.557041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 368/730] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jul 6 13:33:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:33:05.559199 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d478a-4aed-4ecd-9b11-053af4074989 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:05.559487 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d478a-4aed-4ecd-9b11-053af4074989 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:05.559770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d478a-4aed-4ecd-9b11-053af4074989 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:05.565510 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 369/731] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:05.568570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d478a-4aed-4ecd-9b11-053af4074989 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:05.643350 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 363/732] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:33:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 13:33:05.648129 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01233455-200c-41c0-89f8-3450de9f7e65 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:05.649295 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01233455-200c-41c0-89f8-3450de9f7e65 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:05.649504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01233455-200c-41c0-89f8-3450de9f7e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:05.674121 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-01233455-200c-41c0-89f8-3450de9f7e65 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:05.676354 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-01233455-200c-41c0-89f8-3450de9f7e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:05.715191 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d478a-4aed-4ecd-9b11-053af4074989 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:05.735561 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:05.759927 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 364/733] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:33:05.760471 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207bb7ed-9e4d-4012-9310-31ffb49cb094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:05.760879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01233455-200c-41c0-89f8-3450de9f7e65 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:05.761555 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207bb7ed-9e4d-4012-9310-31ffb49cb094 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:05.761967 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207bb7ed-9e4d-4012-9310-31ffb49cb094 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:05.762388 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207bb7ed-9e4d-4012-9310-31ffb49cb094 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:05.763272 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207bb7ed-9e4d-4012-9310-31ffb49cb094 None admin] RBAC: Adding query filter params (name=service) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:05.763578 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207bb7ed-9e4d-4012-9310-31ffb49cb094 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:05.764930 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207bb7ed-9e4d-4012-9310-31ffb49cb094 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:05.773307 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 370/734] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jul 6 13:33:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:05.774551 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d5bf2-ca18-49fd-af89-fca186e416eb None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:05.774841 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d5bf2-ca18-49fd-af89-fca186e416eb None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:05.775124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d5bf2-ca18-49fd-af89-fca186e416eb None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:05.780547 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 371/735] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:05.780547 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d5bf2-ca18-49fd-af89-fca186e416eb None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:05.804993 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:05.875198 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01233455-200c-41c0-89f8-3450de9f7e65 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:05.890792 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d5bf2-ca18-49fd-af89-fca186e416eb None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:05.913851 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e902405-cd7b-47ef-af60-86336fa7fe55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:05.914646 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e902405-cd7b-47ef-af60-86336fa7fe55 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:05.914928 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e902405-cd7b-47ef-af60-86336fa7fe55 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:05.915230 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e902405-cd7b-47ef-af60-86336fa7fe55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:05.926447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463a9100-5a14-4e3f-b790-b1a6b407c785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:05.926610 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e902405-cd7b-47ef-af60-86336fa7fe55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:05.927240 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a9100-5a14-4e3f-b790-b1a6b407c785 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:05.927528 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a9100-5a14-4e3f-b790-b1a6b407c785 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:05.927807 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a9100-5a14-4e3f-b790-b1a6b407c785 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:05.927988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e902405-cd7b-47ef-af60-86336fa7fe55 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:05.940852 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463a9100-5a14-4e3f-b790-b1a6b407c785 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:05.942323 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463a9100-5a14-4e3f-b790-b1a6b407c785 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:05.947498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-3e902405-cd7b-47ef-af60-86336fa7fe55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:05.947498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:05.947498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:05.947498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:05.947498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:05.947498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:05.947498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:05.947498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:05.947498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:05.947498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:05.947498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:05.947498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:05.947498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:05.947498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:05.947498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:05.955189 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:05.955189 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:05.955189 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:05.955189 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:05.955189 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:05.955189 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:05.955189 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:05.955189 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:05.955189 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:05.955189 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:05.955189 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:05.955189 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:05.955189 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:05.955189 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:05.955189 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:05.955189 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:05.955189 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:05.956159 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:05.956159 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:05.956159 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:05.956159 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:05.956159 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 365/736] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:05.956159 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8921e7-c19f-46ef-be27-fcabd87ac105 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:05.956159 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8921e7-c19f-46ef-be27-fcabd87ac105 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:05.956159 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8921e7-c19f-46ef-be27-fcabd87ac105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:05.966516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-463a9100-5a14-4e3f-b790-b1a6b407c785 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:05.966516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:05.966516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:05.966516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:05.966516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:05.966516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:05.966516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:05.966516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:05.966516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:05.966516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:05.966516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:05.966516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:05.966516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:05.966516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:05.966516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:05.967555 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:05.967555 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:05.967555 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:05.967555 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:05.967555 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:05.967555 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:05.967555 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:05.967555 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:05.967555 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:05.967555 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:05.967555 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:05.967555 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:05.967555 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:05.967555 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:05.967555 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:05.967555 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:05.967555 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:05.968462 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:05.968462 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:05.968462 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:05.968462 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:05.968462 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 372/737] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:05.969298 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463a9100-5a14-4e3f-b790-b1a6b407c785 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:05.974859 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-ca8921e7-c19f-46ef-be27-fcabd87ac105 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:05.976045 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-ca8921e7-c19f-46ef-be27-fcabd87ac105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:06.005110 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:06.047584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:06.049189 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 366/738] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:33:06.050213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8921e7-c19f-46ef-be27-fcabd87ac105 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:06.110362 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463a9100-5a14-4e3f-b790-b1a6b407c785 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:06.146049 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2570d-8864-43a9-9a5c-b3985ad28714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:06.146950 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2570d-8864-43a9-9a5c-b3985ad28714 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:06.147240 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2570d-8864-43a9-9a5c-b3985ad28714 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:06.147516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2570d-8864-43a9-9a5c-b3985ad28714 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:06.159222 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d2570d-8864-43a9-9a5c-b3985ad28714 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:33:06.163709 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8921e7-c19f-46ef-be27-fcabd87ac105 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:06.167412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-24d2570d-8864-43a9-9a5c-b3985ad28714 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:33:06.167412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:06.167412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:06.167412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:06.167412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:06.167412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:06.167412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:06.167412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:06.167412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:06.167412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:06.167412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:06.167412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:06.167412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:33:06.167412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:33:06.167412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:33:06.167412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:33:06.168508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:33:06.168508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:33:06.168508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:33:06.168508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:33:06.168508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:06.168508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:06.168508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:06.168508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:06.168508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:06.168508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:06.168508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:06.168508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:06.168508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:06.168508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:06.168508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:06.168508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:06.169530 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:06.169530 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:06.169530 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:06.169530 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:06.169530 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:33:06.169530 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:33:06.169530 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:33:06.169530 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:33:06.169530 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:33:06.169530 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:33:06.169530 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:33:06.169530 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:06.173302 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 373/739] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:33:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:06.174677 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2570d-8864-43a9-9a5c-b3985ad28714 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:06.204573 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eca6be-6a8a-4bd6-8ef1-b777c2d2ee74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:06.205284 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eca6be-6a8a-4bd6-8ef1-b777c2d2ee74 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:06.205560 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eca6be-6a8a-4bd6-8ef1-b777c2d2ee74 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:06.205840 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eca6be-6a8a-4bd6-8ef1-b777c2d2ee74 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:06.206567 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eca6be-6a8a-4bd6-8ef1-b777c2d2ee74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:06.206877 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eca6be-6a8a-4bd6-8ef1-b777c2d2ee74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:06.208414 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eca6be-6a8a-4bd6-8ef1-b777c2d2ee74 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:06.221370 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 367/740] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:06.222066 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eca6be-6a8a-4bd6-8ef1-b777c2d2ee74 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:06.248584 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:06.296621 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2570d-8864-43a9-9a5c-b3985ad28714 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:06.326707 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eca6be-6a8a-4bd6-8ef1-b777c2d2ee74 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:06.329284 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:06.334776 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdeaab-7a9c-4e02-a14c-df682b9147f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:06.335459 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdeaab-7a9c-4e02-a14c-df682b9147f4 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:06.335804 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdeaab-7a9c-4e02-a14c-df682b9147f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:06.336298 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdeaab-7a9c-4e02-a14c-df682b9147f4 None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:06.337053 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fdeaab-7a9c-4e02-a14c-df682b9147f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:06.337363 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fdeaab-7a9c-4e02-a14c-df682b9147f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:06.339387 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fdeaab-7a9c-4e02-a14c-df682b9147f4 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:06.357115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 374/741] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:06.357115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdeaab-7a9c-4e02-a14c-df682b9147f4 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:06.374257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc25dad-e822-4715-9a87-58d52037ce42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:06.374938 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc25dad-e822-4715-9a87-58d52037ce42 None admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:06.375236 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc25dad-e822-4715-9a87-58d52037ce42 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:06.375502 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc25dad-e822-4715-9a87-58d52037ce42 None admin] PATH_INFO: `/v3/services` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:06.376233 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc25dad-e822-4715-9a87-58d52037ce42 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:06.377671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc25dad-e822-4715-9a87-58d52037ce42 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:06.394919 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 368/742] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Jul 6 13:33:06 2022] POST /identity/v3/services => generated 251 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:33:06.395539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc25dad-e822-4715-9a87-58d52037ce42 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:06.499637 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdeaab-7a9c-4e02-a14c-df682b9147f4 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:06.501697 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:06.535752 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc25dad-e822-4715-9a87-58d52037ce42 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:06.536085 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4f8e8-6b33-4b9e-b374-ed58ed6641f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:06.536731 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4f8e8-6b33-4b9e-b374-ed58ed6641f7 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:06.537038 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4f8e8-6b33-4b9e-b374-ed58ed6641f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:06.537378 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4f8e8-6b33-4b9e-b374-ed58ed6641f7 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:06.539355 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be4f8e8-6b33-4b9e-b374-ed58ed6641f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:06.539689 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be4f8e8-6b33-4b9e-b374-ed58ed6641f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:06.541110 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be4f8e8-6b33-4b9e-b374-ed58ed6641f7 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:06.551957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 375/743] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:33:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:06.553662 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4f8e8-6b33-4b9e-b374-ed58ed6641f7 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:06.579764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec968778-5f91-418d-b6e5-a32f4fdf60a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:06.580496 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec968778-5f91-418d-b6e5-a32f4fdf60a2 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:06.580780 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec968778-5f91-418d-b6e5-a32f4fdf60a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:06.581562 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec968778-5f91-418d-b6e5-a32f4fdf60a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:06.588812 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec968778-5f91-418d-b6e5-a32f4fdf60a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:06.590402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec968778-5f91-418d-b6e5-a32f4fdf60a2 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:06.605906 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-ec968778-5f91-418d-b6e5-a32f4fdf60a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:06.605906 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:06.605906 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:06.605906 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:06.605906 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:06.605906 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:06.605906 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:06.605906 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:06.605906 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:06.605906 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:06.605906 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:06.605906 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:06.605906 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:33:06.605906 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:33:06.605906 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:33:06.607028 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:33:06.607028 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:06.607028 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:06.607028 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:06.607028 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:06.607028 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:06.607028 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:06.607028 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:06.607028 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:06.607028 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:06.607028 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:06.607028 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:06.607028 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:06.607028 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:06.607028 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:06.607028 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:06.607028 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:06.608061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:33:06.608061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:33:06.608061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:33:06.608061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:33:06.608061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:33:06.608061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:33:06.608061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:06.608061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:06.608061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 369/744] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:33:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:06.608622 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c1d5d-151f-4eae-b11c-28264dd50156 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:06.608622 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c1d5d-151f-4eae-b11c-28264dd50156 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:06.608864 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c1d5d-151f-4eae-b11c-28264dd50156 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:06.617486 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 370/745] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:06.617486 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c1d5d-151f-4eae-b11c-28264dd50156 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:06.621243 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:06.670965 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4f8e8-6b33-4b9e-b374-ed58ed6641f7 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:06.707561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5b95b-bfa2-4770-b9c5-cef6b1e68def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:06.708630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5b95b-bfa2-4770-b9c5-cef6b1e68def None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:06.709105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5b95b-bfa2-4770-b9c5-cef6b1e68def None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:06.709385 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5b95b-bfa2-4770-b9c5-cef6b1e68def None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:06.717721 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5b95b-bfa2-4770-b9c5-cef6b1e68def None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:06.719320 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5b95b-bfa2-4770-b9c5-cef6b1e68def None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:06.728772 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c1d5d-151f-4eae-b11c-28264dd50156 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:06.735833 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-16c5b95b-bfa2-4770-b9c5-cef6b1e68def None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 13:33:06.735833 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:06.735833 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:06.735833 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:06.735833 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:06.735833 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:06.735833 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:06.735833 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:06.735833 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:06.735833 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:06.735833 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:06.735833 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:06.735833 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:33:06.735833 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:33:06.735833 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:33:06.735833 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:33:06.736952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:06.736952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:06.736952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:33:06.736952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:06.736952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:33:06.736952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:06.736952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:06.736952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:06.736952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:06.736952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:06.736952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:06.736952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:06.736952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:06.736952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:06.736952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:06.736952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:06.736952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:06.736952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:06.739860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:06.739860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:06.739860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:33:06.739860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:33:06.739860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:33:06.739860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:33:06.739860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:33:06.739860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:33:06.739860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 13:33:06.739860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:06.739860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 376/746] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Jul 6 13:33:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:06.739860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5b95b-bfa2-4770-b9c5-cef6b1e68def None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:06.769817 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb0cd78-4625-4b93-b725-03901559c8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:06.770623 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0cd78-4625-4b93-b725-03901559c8a5 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:06.770900 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0cd78-4625-4b93-b725-03901559c8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:06.771187 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0cd78-4625-4b93-b725-03901559c8a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:06.778034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:06.782462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb0cd78-4625-4b93-b725-03901559c8a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:06.783907 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb0cd78-4625-4b93-b725-03901559c8a5 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:06.801443 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-ceb0cd78-4625-4b93-b725-03901559c8a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:06.801443 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:06.801443 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:06.801443 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:06.801443 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:06.801443 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:06.801443 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:06.801443 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:06.801443 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:06.801443 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:06.801443 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:06.801443 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:06.801443 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:06.801443 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:06.801443 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:06.802519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:06.802519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:06.802519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:06.802519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:06.802519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:06.802519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:06.802519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:06.802519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:06.802519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:06.802519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:06.802519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:06.802519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:06.802519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:06.802519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:06.802519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:06.802519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:06.802519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:06.803459 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:06.803459 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:06.803459 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:06.803459 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:06.803459 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 371/747] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:06.803804 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731ac89-b86c-4a2b-a575-30b314e24610 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:06.804088 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731ac89-b86c-4a2b-a575-30b314e24610 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:06.804362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731ac89-b86c-4a2b-a575-30b314e24610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:06.820511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-8731ac89-b86c-4a2b-a575-30b314e24610 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:06.821706 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-8731ac89-b86c-4a2b-a575-30b314e24610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:06.857933 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5b95b-bfa2-4770-b9c5-cef6b1e68def None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:06.872367 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:06.894533 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bea200-9470-4a30-9612-4229d317254d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:06.895199 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bea200-9470-4a30-9612-4229d317254d None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:06.895486 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bea200-9470-4a30-9612-4229d317254d None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:06.895765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bea200-9470-4a30-9612-4229d317254d None admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:06.896566 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bea200-9470-4a30-9612-4229d317254d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:06.896878 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bea200-9470-4a30-9612-4229d317254d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:06.898332 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bea200-9470-4a30-9612-4229d317254d None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:06.900120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 372/748] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:06 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 13:33:06.901850 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731ac89-b86c-4a2b-a575-30b314e24610 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:06.914475 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 377/749] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:33:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:06.915864 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bea200-9470-4a30-9612-4229d317254d None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:07.012444 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731ac89-b86c-4a2b-a575-30b314e24610 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:07.019423 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bea200-9470-4a30-9612-4229d317254d None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:07.034694 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:07.049864 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be591400-491a-4fa6-b391-1124afe6e87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:07.050546 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be591400-491a-4fa6-b391-1124afe6e87e None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:07.050819 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be591400-491a-4fa6-b391-1124afe6e87e None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:07.051099 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be591400-491a-4fa6-b391-1124afe6e87e None admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:07.052005 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be591400-491a-4fa6-b391-1124afe6e87e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:07.052316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be591400-491a-4fa6-b391-1124afe6e87e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:07.053778 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be591400-491a-4fa6-b391-1124afe6e87e None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:07.055890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46886772-168a-4315-bd04-fc587e116747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:07.056557 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46886772-168a-4315-bd04-fc587e116747 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:07.056836 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46886772-168a-4315-bd04-fc587e116747 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:07.057689 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46886772-168a-4315-bd04-fc587e116747 None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:07.058410 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46886772-168a-4315-bd04-fc587e116747 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:07.058718 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46886772-168a-4315-bd04-fc587e116747 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:07.060055 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46886772-168a-4315-bd04-fc587e116747 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:07.071555 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 378/750] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:07.072338 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46886772-168a-4315-bd04-fc587e116747 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:07.073638 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 373/751] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Jul 6 13:33:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:07.074694 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be591400-491a-4fa6-b391-1124afe6e87e None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:07.112751 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:07.182929 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46886772-168a-4315-bd04-fc587e116747 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:07.184305 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be591400-491a-4fa6-b391-1124afe6e87e None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:07.218939 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e305283-a9df-4dda-8ed9-64dafbb2d58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:07.219330 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb5c63-9b11-49a8-88a7-235b6260084a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:07.219605 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e305283-a9df-4dda-8ed9-64dafbb2d58a None admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:07.219887 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e305283-a9df-4dda-8ed9-64dafbb2d58a None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:07.219985 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb5c63-9b11-49a8-88a7-235b6260084a None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:07.220167 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e305283-a9df-4dda-8ed9-64dafbb2d58a None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:07.220270 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb5c63-9b11-49a8-88a7-235b6260084a None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:07.220524 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb5c63-9b11-49a8-88a7-235b6260084a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:07.220883 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e305283-a9df-4dda-8ed9-64dafbb2d58a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:07.221392 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb5c63-9b11-49a8-88a7-235b6260084a None admin] RBAC: Adding query filter params (role.id=795daa2c78354fd48e4a741e1182cc98, scope.project.id=bfe6c3d8da2743be9609184db2506303, user.id=8c1bea05c37646b0a05fa8dda0367ef2) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:07.221712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb5c63-9b11-49a8-88a7-235b6260084a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:07.222281 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e305283-a9df-4dda-8ed9-64dafbb2d58a None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:07.223142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb5c63-9b11-49a8-88a7-235b6260084a None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:07.229475 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 374/752] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:33:07 2022] GET /identity/v3/role_assignments?user.id=8c1bea05c37646b0a05fa8dda0367ef2&scope.project.id=bfe6c3d8da2743be9609184db2506303&role.id=795daa2c78354fd48e4a741e1182cc98 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:07.230069 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb5c63-9b11-49a8-88a7-235b6260084a None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:07.257069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 379/753] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Jul 6 13:33:07 2022] POST /identity/v3/services => generated 240 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:33:07.258110 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e305283-a9df-4dda-8ed9-64dafbb2d58a None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:07.357566 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:07.416486 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:07.455806 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e305283-a9df-4dda-8ed9-64dafbb2d58a None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:07.468878 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb5c63-9b11-49a8-88a7-235b6260084a None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:07.511396 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ea7ee-9b8a-416e-b524-7dd7b18f0460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:07.512063 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ea7ee-9b8a-416e-b524-7dd7b18f0460 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:07.512346 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ea7ee-9b8a-416e-b524-7dd7b18f0460 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:07.512627 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ea7ee-9b8a-416e-b524-7dd7b18f0460 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:07.515753 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6133db2-0560-42e6-8ecf-85a1a1c01f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:07.516844 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6133db2-0560-42e6-8ecf-85a1a1c01f74 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:07.517169 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6133db2-0560-42e6-8ecf-85a1a1c01f74 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:07.517450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6133db2-0560-42e6-8ecf-85a1a1c01f74 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:07.529608 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6133db2-0560-42e6-8ecf-85a1a1c01f74 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:07.531834 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6133db2-0560-42e6-8ecf-85a1a1c01f74 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:07.551859 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-d6133db2-0560-42e6-8ecf-85a1a1c01f74 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 06 13:33:07.551859 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:07.551859 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:07.551859 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:07.551859 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:07.551859 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:07.551859 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:07.551859 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:07.551859 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:07.551859 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:07.551859 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:07.551859 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:07.551859 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:33:07.551859 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:33:07.551859 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:33:07.551859 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:33:07.552846 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:07.552846 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:07.552846 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:33:07.552846 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:07.552846 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:33:07.552846 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:07.552846 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:07.552846 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:07.552846 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:07.552846 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:07.552846 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:07.552846 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:07.552846 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:07.552846 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:07.552846 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:07.552846 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:07.552846 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:07.552846 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:07.553820 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:07.553820 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:07.553820 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:33:07.553820 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:33:07.553820 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:33:07.553820 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:33:07.553820 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:33:07.553820 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:33:07.553820 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 06 13:33:07.553820 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:07.553820 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 375/754] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Jul 6 13:33:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:07.560224 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6133db2-0560-42e6-8ecf-85a1a1c01f74 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:07.642060 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ea7ee-9b8a-416e-b524-7dd7b18f0460 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:07.643570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ea7ee-9b8a-416e-b524-7dd7b18f0460 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:07.813356 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6133db2-0560-42e6-8ecf-85a1a1c01f74 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:07.817892 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:07.874949 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:07.880524 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccfc44c-1757-4965-b515-9e076113e273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:07.881230 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfc44c-1757-4965-b515-9e076113e273 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:07.881515 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfc44c-1757-4965-b515-9e076113e273 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:07.881783 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfc44c-1757-4965-b515-9e076113e273 None admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:07.882623 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccfc44c-1757-4965-b515-9e076113e273 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:07.882934 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccfc44c-1757-4965-b515-9e076113e273 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:07.885629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccfc44c-1757-4965-b515-9e076113e273 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:07.893527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 380/755] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:33:07 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 13:33:07.904308 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ea7ee-9b8a-416e-b524-7dd7b18f0460 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:07.914754 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 376/756] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Jul 6 13:33:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:07.920607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccfc44c-1757-4965-b515-9e076113e273 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:08.084376 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccfc44c-1757-4965-b515-9e076113e273 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:08.112655 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ea7ee-9b8a-416e-b524-7dd7b18f0460 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:08.127592 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93226d-df11-4f43-83e6-b8f174eb32c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:08.128390 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93226d-df11-4f43-83e6-b8f174eb32c2 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:08.128671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93226d-df11-4f43-83e6-b8f174eb32c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:08.128943 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93226d-df11-4f43-83e6-b8f174eb32c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:08.154048 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f93226d-df11-4f43-83e6-b8f174eb32c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:08.155523 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f93226d-df11-4f43-83e6-b8f174eb32c2 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:08.176192 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20340b5-314d-4023-8bb1-d89e58436d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:08.176989 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20340b5-314d-4023-8bb1-d89e58436d47 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:08.177296 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20340b5-314d-4023-8bb1-d89e58436d47 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:08.177558 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20340b5-314d-4023-8bb1-d89e58436d47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:08.194570 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-0f93226d-df11-4f43-83e6-b8f174eb32c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:08.194570 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:08.194570 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:08.194570 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:08.194570 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:08.194570 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:08.194570 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:08.194570 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:08.194570 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:08.194570 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:08.194570 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:08.194570 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:08.194570 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:08.194570 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:08.194570 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:08.195510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:08.195510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:08.195510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:08.195510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:08.195510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:08.195510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:08.195510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:08.195510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:08.195510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:08.195510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:08.195510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:08.195510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:08.195510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:08.195510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:08.195510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:08.195510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:08.195510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:08.196392 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:08.196392 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:08.196392 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:08.196392 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:08.200900 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 377/757] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:08.201153 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20340b5-314d-4023-8bb1-d89e58436d47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:08.202626 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20340b5-314d-4023-8bb1-d89e58436d47 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:08.207752 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93226d-df11-4f43-83e6-b8f174eb32c2 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:08.237363 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:08.243136 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-d20340b5-314d-4023-8bb1-d89e58436d47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:08.243136 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:08.243136 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:08.243136 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:08.243136 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:08.243136 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:08.243136 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:08.243136 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:08.243136 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:08.243136 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:08.243136 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:08.243136 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:08.243136 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:08.243136 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:08.243136 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:08.245418 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:08.245418 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:08.245418 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:08.245418 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:08.245418 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:08.245418 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:08.245418 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:08.245418 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:08.245418 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:08.245418 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:08.245418 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:08.245418 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:08.245418 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:08.245418 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:08.245418 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:08.245418 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:08.245418 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:08.250782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:08.250782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:08.250782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:08.250782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:08.250782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 381/758] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:08.252408 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20340b5-314d-4023-8bb1-d89e58436d47 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:08.349246 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:08.398375 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93226d-df11-4f43-83e6-b8f174eb32c2 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:08.458478 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20340b5-314d-4023-8bb1-d89e58436d47 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:08.466119 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762960e-aea7-4a35-8e80-f66874026b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:08.471013 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762960e-aea7-4a35-8e80-f66874026b4d None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:08.471941 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762960e-aea7-4a35-8e80-f66874026b4d None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:08.472404 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762960e-aea7-4a35-8e80-f66874026b4d None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:08.473355 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762960e-aea7-4a35-8e80-f66874026b4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:08.473849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762960e-aea7-4a35-8e80-f66874026b4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:08.475654 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762960e-aea7-4a35-8e80-f66874026b4d None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:08.504010 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 378/759] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:08.505207 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762960e-aea7-4a35-8e80-f66874026b4d None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:08.533300 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf1741-8c28-4a7d-8fa5-d8b28313eec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:08.534266 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf1741-8c28-4a7d-8fa5-d8b28313eec6 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:08.534918 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf1741-8c28-4a7d-8fa5-d8b28313eec6 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:08.536060 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf1741-8c28-4a7d-8fa5-d8b28313eec6 None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:08.537417 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf1741-8c28-4a7d-8fa5-d8b28313eec6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:08.537915 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf1741-8c28-4a7d-8fa5-d8b28313eec6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:08.539477 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf1741-8c28-4a7d-8fa5-d8b28313eec6 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:08.553258 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 382/760] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:08.554997 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c202ca5-9fe3-47a4-bc4f-e4c813dcf918 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:08.555396 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c202ca5-9fe3-47a4-bc4f-e4c813dcf918 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:08.555776 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c202ca5-9fe3-47a4-bc4f-e4c813dcf918 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:08.558201 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 383/761] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:08.559028 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c202ca5-9fe3-47a4-bc4f-e4c813dcf918 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:08.601795 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:08.637461 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762960e-aea7-4a35-8e80-f66874026b4d None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:08.674488 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c202ca5-9fe3-47a4-bc4f-e4c813dcf918 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:08.676826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:08.679869 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600dce49-46a6-4bf4-a663-264d86c5f483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:08.680768 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600dce49-46a6-4bf4-a663-264d86c5f483 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:08.681156 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600dce49-46a6-4bf4-a663-264d86c5f483 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:08.681435 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600dce49-46a6-4bf4-a663-264d86c5f483 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:08.690439 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600dce49-46a6-4bf4-a663-264d86c5f483 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:08.692001 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600dce49-46a6-4bf4-a663-264d86c5f483 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:08.707814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-600dce49-46a6-4bf4-a663-264d86c5f483 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:08.707814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:08.707814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:08.707814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:08.707814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:08.707814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:08.707814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:08.707814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:08.707814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:08.707814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:08.707814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:08.707814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:08.707814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:33:08.707814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:33:08.707814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:33:08.708755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:33:08.708755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:08.708755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:08.708755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:08.708755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:08.708755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:08.708755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:08.708755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:08.708755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:08.708755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:08.708755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:08.708755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:08.708755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:08.708755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:08.708755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:08.708755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:08.708755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:08.710266 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:33:08.710266 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:33:08.710266 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:33:08.710266 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:33:08.710266 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:33:08.710266 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:33:08.710266 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:08.710266 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:08.710266 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 379/762] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:33:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:08.710740 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7febed0b-815f-4aaf-88da-417715a63132 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:08.710996 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7febed0b-815f-4aaf-88da-417715a63132 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:08.711148 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7febed0b-815f-4aaf-88da-417715a63132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:08.717962 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6daac7d-d06d-453b-9b75-61228a59dd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:08.719110 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6daac7d-d06d-453b-9b75-61228a59dd1a None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:08.719626 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6daac7d-d06d-453b-9b75-61228a59dd1a None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:08.720094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6daac7d-d06d-453b-9b75-61228a59dd1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:08.727390 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6daac7d-d06d-453b-9b75-61228a59dd1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:08.729698 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6daac7d-d06d-453b-9b75-61228a59dd1a None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:08.730477 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-7febed0b-815f-4aaf-88da-417715a63132 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:08.731773 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-7febed0b-815f-4aaf-88da-417715a63132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:08.745554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-c6daac7d-d06d-453b-9b75-61228a59dd1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:08.745554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:08.745554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:08.745554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:08.745554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:08.745554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:08.745554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:08.745554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:08.745554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:08.745554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:08.745554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:08.745554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:08.745554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:33:08.745554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:33:08.745554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:33:08.746472 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:33:08.746472 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:08.746472 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:08.746472 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:08.746472 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:08.746472 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:08.746472 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:08.746472 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:08.746472 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:08.746472 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:08.746472 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:08.746472 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:08.746472 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:08.746472 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:08.746472 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:08.746472 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:08.746472 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:08.747383 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:33:08.747383 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:33:08.747383 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:33:08.747383 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:33:08.747383 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:33:08.747383 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:33:08.747383 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:08.747383 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:08.750257 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 384/763] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:33:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:08.751310 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6daac7d-d06d-453b-9b75-61228a59dd1a None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:08.813079 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 380/764] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:08.814254 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7febed0b-815f-4aaf-88da-417715a63132 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:08.875857 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6daac7d-d06d-453b-9b75-61228a59dd1a None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:08.904764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:08.921905 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb65eb-9f6f-4c7b-9338-77e00c3196fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:08.922607 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb65eb-9f6f-4c7b-9338-77e00c3196fc None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:08.922898 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb65eb-9f6f-4c7b-9338-77e00c3196fc None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:08.923182 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb65eb-9f6f-4c7b-9338-77e00c3196fc None admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:08.924649 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb65eb-9f6f-4c7b-9338-77e00c3196fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:08.924970 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb65eb-9f6f-4c7b-9338-77e00c3196fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:08.926693 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb65eb-9f6f-4c7b-9338-77e00c3196fc None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:08.945306 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 385/765] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:33:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:08.946500 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb65eb-9f6f-4c7b-9338-77e00c3196fc None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:08.949146 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7febed0b-815f-4aaf-88da-417715a63132 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:08.973784 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:09.004261 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27b399-aeca-49a2-9696-4fdc0841831e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:09.004938 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27b399-aeca-49a2-9696-4fdc0841831e None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:09.005255 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27b399-aeca-49a2-9696-4fdc0841831e None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:09.005533 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27b399-aeca-49a2-9696-4fdc0841831e None admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:09.006330 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b27b399-aeca-49a2-9696-4fdc0841831e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:09.006641 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b27b399-aeca-49a2-9696-4fdc0841831e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:09.008932 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b27b399-aeca-49a2-9696-4fdc0841831e None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:09.027617 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 381/766] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:33:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:09.028549 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27b399-aeca-49a2-9696-4fdc0841831e None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:09.060531 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb65eb-9f6f-4c7b-9338-77e00c3196fc None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:09.098219 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b24bc-4714-4c61-a53f-2e5dcb9043c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:09.099424 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b24bc-4714-4c61-a53f-2e5dcb9043c4 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:09.099903 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b24bc-4714-4c61-a53f-2e5dcb9043c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:09.100472 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b24bc-4714-4c61-a53f-2e5dcb9043c4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:09.101349 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b24bc-4714-4c61-a53f-2e5dcb9043c4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:09.102950 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b24bc-4714-4c61-a53f-2e5dcb9043c4 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:09.118164 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-ad5b24bc-4714-4c61-a53f-2e5dcb9043c4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 06 13:33:09.118164 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:09.118164 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 13:33:09.118164 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 13:33:09.118164 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 13:33:09.118164 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 13:33:09.118164 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 13:33:09.118164 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:09.118164 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 06 13:33:09.118164 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:09.118164 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 13:33:09.118164 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:09.118164 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:09.118164 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:09.118164 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:09.118164 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:09.119145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:09.119145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:09.119145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:09.119145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:09.119145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:09.119145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:09.119145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:09.119145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 13:33:09.119145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 13:33:09.119145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 13:33:09.119145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 13:33:09.119145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:09.119145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:09.119145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:09.119145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:09.119145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:09.120561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:09.120561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:09.120561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:09.120561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:09.120561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:09.120561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:09.120561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:09.120561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:09.120561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:09.120561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:09.120561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:09.120561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 13:33:09.120561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:09.120561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 06 13:33:09.120561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:09.128490 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 386/767] 10.222.0.20 () {56 vars in 1129 bytes} [Wed Jul 6 13:33:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:09.129660 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b24bc-4714-4c61-a53f-2e5dcb9043c4 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:09.159356 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27b399-aeca-49a2-9696-4fdc0841831e None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:09.185958 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:09.198894 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb202c-4fbd-4a4c-b41a-8fb4e3d5166c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:09.200115 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb202c-4fbd-4a4c-b41a-8fb4e3d5166c None admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:09.200598 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb202c-4fbd-4a4c-b41a-8fb4e3d5166c None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:09.201163 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb202c-4fbd-4a4c-b41a-8fb4e3d5166c None admin] PATH_INFO: `/v3/projects/bfe6c3d8da2743be9609184db2506303/users/6647eb1e8a1544ef9d3a9806f2d1fc51/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:09.203525 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbb202c-4fbd-4a4c-b41a-8fb4e3d5166c None admin] RBAC: Authorizing `identity:create_grant(project_id=bfe6c3d8da2743be9609184db2506303, user_id=6647eb1e8a1544ef9d3a9806f2d1fc51, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:09.205396 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbb202c-4fbd-4a4c-b41a-8fb4e3d5166c None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:09.223761 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 382/768] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jul 6 13:33:09 2022] PUT /identity/v3/projects/bfe6c3d8da2743be9609184db2506303/users/6647eb1e8a1544ef9d3a9806f2d1fc51/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:33:09.225045 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb202c-4fbd-4a4c-b41a-8fb4e3d5166c None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:09.239715 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:09.259590 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b24bc-4714-4c61-a53f-2e5dcb9043c4 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:09.318182 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be48b8bd-8403-4e52-a408-c4ebba9b7859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:09.319241 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be48b8bd-8403-4e52-a408-c4ebba9b7859 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:09.319841 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be48b8bd-8403-4e52-a408-c4ebba9b7859 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:09.320263 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be48b8bd-8403-4e52-a408-c4ebba9b7859 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:09.321551 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be48b8bd-8403-4e52-a408-c4ebba9b7859 None admin] RBAC: Adding query filter params (role.id=a6853d485f7942259b3646de3f386852, scope.project.id=bfe6c3d8da2743be9609184db2506303, user.id=c4535134181e489fb532f617264d5855) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:09.323046 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be48b8bd-8403-4e52-a408-c4ebba9b7859 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:09.324689 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be48b8bd-8403-4e52-a408-c4ebba9b7859 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:09.333037 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 387/769] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:33:09 2022] GET /identity/v3/role_assignments?user.id=c4535134181e489fb532f617264d5855&scope.project.id=bfe6c3d8da2743be9609184db2506303&role.id=a6853d485f7942259b3646de3f386852 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:09.365265 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb202c-4fbd-4a4c-b41a-8fb4e3d5166c None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:09.408209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9616002-be62-4efc-9bc9-c00073177b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:09.408862 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9616002-be62-4efc-9bc9-c00073177b75 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:09.409457 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9616002-be62-4efc-9bc9-c00073177b75 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:09.409977 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9616002-be62-4efc-9bc9-c00073177b75 None admin] PATH_INFO: `/v3/services` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:09.411369 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9616002-be62-4efc-9bc9-c00073177b75 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:09.411857 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9616002-be62-4efc-9bc9-c00073177b75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:09.413353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9616002-be62-4efc-9bc9-c00073177b75 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:09.420176 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 383/770] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Jul 6 13:33:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:09.426683 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be48b8bd-8403-4e52-a408-c4ebba9b7859 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:09.567346 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be48b8bd-8403-4e52-a408-c4ebba9b7859 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:09.607497 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e19f86-0fa3-4ca1-98ee-c4f79f7dcb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:09.608538 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e19f86-0fa3-4ca1-98ee-c4f79f7dcb29 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:09.609084 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e19f86-0fa3-4ca1-98ee-c4f79f7dcb29 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:09.609562 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e19f86-0fa3-4ca1-98ee-c4f79f7dcb29 None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:09.610469 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e19f86-0fa3-4ca1-98ee-c4f79f7dcb29 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:09.610991 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e19f86-0fa3-4ca1-98ee-c4f79f7dcb29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:09.612560 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e19f86-0fa3-4ca1-98ee-c4f79f7dcb29 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:09.618435 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 388/771] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Jul 6 13:33:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:09.624486 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9616002-be62-4efc-9bc9-c00073177b75 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:09.642543 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2898b3-e809-48f9-9f99-76423ffdb9e9 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:09.643194 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2898b3-e809-48f9-9f99-76423ffdb9e9 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:09.643701 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2898b3-e809-48f9-9f99-76423ffdb9e9 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:09.645497 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 389/772] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:09.651868 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae60804-10ae-4895-820c-f4f3f176476b None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:09.652390 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae60804-10ae-4895-820c-f4f3f176476b None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:09.652930 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae60804-10ae-4895-820c-f4f3f176476b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:09.690685 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-5ae60804-10ae-4895-820c-f4f3f176476b None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:09.693041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-5ae60804-10ae-4895-820c-f4f3f176476b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:09.702831 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:09.729345 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:09.789160 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 390/773] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:09.792197 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837908b-afe2-4fb2-9067-bf62f465612c None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:09.792650 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837908b-afe2-4fb2-9067-bf62f465612c None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:09.793035 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837908b-afe2-4fb2-9067-bf62f465612c None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:09.794681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 391/774] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:09.802396 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac589b-b9b3-46d0-bb54-8b01594f3167 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:09.802814 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac589b-b9b3-46d0-bb54-8b01594f3167 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:09.803202 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac589b-b9b3-46d0-bb54-8b01594f3167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:09.818351 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9616002-be62-4efc-9bc9-c00073177b75 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:09.828936 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-9bac589b-b9b3-46d0-bb54-8b01594f3167 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:09.830352 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-9bac589b-b9b3-46d0-bb54-8b01594f3167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:09.854362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f60e0-b897-447a-91ab-ebe7b68c15e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:09.855106 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f60e0-b897-447a-91ab-ebe7b68c15e0 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:09.855393 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f60e0-b897-447a-91ab-ebe7b68c15e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:09.855671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f60e0-b897-447a-91ab-ebe7b68c15e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:09.856523 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187f60e0-b897-447a-91ab-ebe7b68c15e0 None admin] RBAC: Adding query filter params (interface=public, service_id=7631a495b4ff40d39802a5e80b08843b, region_id=RegionOne) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:09.856843 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187f60e0-b897-447a-91ab-ebe7b68c15e0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:09.858244 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187f60e0-b897-447a-91ab-ebe7b68c15e0 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:09.866460 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 384/775] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Jul 6 13:33:09 2022] GET /identity/v3/endpoints?service_id=7631a495b4ff40d39802a5e80b08843b&interface=public®ion_id=RegionOne => generated 194 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:09.867543 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f60e0-b897-447a-91ab-ebe7b68c15e0 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:09.907604 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 392/776] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 13:33:09.908682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac589b-b9b3-46d0-bb54-8b01594f3167 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:09.973770 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f60e0-b897-447a-91ab-ebe7b68c15e0 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:09.984358 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:10.008960 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd6d4f-b963-4769-adba-45511d29afa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:10.009533 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd6d4f-b963-4769-adba-45511d29afa0 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:10.009812 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd6d4f-b963-4769-adba-45511d29afa0 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:10.010784 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd6d4f-b963-4769-adba-45511d29afa0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:10.020686 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:10.022812 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cd6d4f-b963-4769-adba-45511d29afa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:33:10.030711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-78cd6d4f-b963-4769-adba-45511d29afa0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:33:10.030711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:10.030711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:10.030711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:10.030711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:10.030711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:10.030711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:10.030711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:10.030711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:10.030711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:10.030711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:10.030711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:10.030711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:33:10.030711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:33:10.030711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:33:10.030711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:33:10.031848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:33:10.031848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:33:10.031848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:33:10.031848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:33:10.031848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:10.031848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:10.031848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:10.031848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:10.031848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:10.031848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:10.031848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:10.031848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:10.031848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:10.031848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:10.031848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:10.031848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:10.032811 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:10.032811 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:10.032811 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:10.032811 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:10.032811 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:33:10.032811 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:33:10.032811 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:33:10.032811 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:33:10.032811 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:33:10.032811 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:33:10.032811 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:33:10.032811 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:10.032811 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 385/777] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:33:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:10.033543 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd6d4f-b963-4769-adba-45511d29afa0 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:10.051211 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac589b-b9b3-46d0-bb54-8b01594f3167 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:10.092132 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b4c114-6cb6-4a1c-bdb7-b626320f389c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:10.093093 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4c114-6cb6-4a1c-bdb7-b626320f389c None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:10.093630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4c114-6cb6-4a1c-bdb7-b626320f389c None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:10.094262 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4c114-6cb6-4a1c-bdb7-b626320f389c None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:10.095773 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b4c114-6cb6-4a1c-bdb7-b626320f389c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:10.097627 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b4c114-6cb6-4a1c-bdb7-b626320f389c None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:10.105655 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 393/778] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Jul 6 13:33:09 2022] GET /identity/v3/services => generated 772 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:10.106745 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b4c114-6cb6-4a1c-bdb7-b626320f389c None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:10.148049 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd6d4f-b963-4769-adba-45511d29afa0 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:10.185888 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd2108a-75e3-4f50-922d-bb3f90641f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:10.186697 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2108a-75e3-4f50-922d-bb3f90641f44 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:10.186980 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2108a-75e3-4f50-922d-bb3f90641f44 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:10.187307 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2108a-75e3-4f50-922d-bb3f90641f44 None admin] PATH_INFO: `/v3/services/network` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:10.187912 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd2108a-75e3-4f50-922d-bb3f90641f44 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:10.189406 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd2108a-75e3-4f50-922d-bb3f90641f44 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:10.202954 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-7cd2108a-75e3-4f50-922d-bb3f90641f44 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 06 13:33:10.202954 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:10.202954 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 13:33:10.202954 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 13:33:10.202954 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 13:33:10.202954 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 13:33:10.202954 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 13:33:10.202954 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:10.202954 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 06 13:33:10.202954 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:10.202954 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 13:33:10.202954 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:10.202954 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:10.202954 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:10.202954 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:10.202954 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:10.202954 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:10.204020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:10.204020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:10.204020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:10.204020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:10.204020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:10.204020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:10.204020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 13:33:10.204020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 13:33:10.204020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 13:33:10.204020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 13:33:10.204020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:10.204020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:10.204020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:10.204020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:10.204020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:10.204020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:10.204938 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:10.204938 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:10.204938 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:10.204938 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:10.204938 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:10.204938 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:10.204938 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:10.204938 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:10.204938 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:10.204938 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:10.204938 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 13:33:10.204938 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:10.204938 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 06 13:33:10.204938 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:10.204938 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 386/779] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:33:10 2022] GET /identity/v3/services/network => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:10.214943 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd2108a-75e3-4f50-922d-bb3f90641f44 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:10.229660 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b4c114-6cb6-4a1c-bdb7-b626320f389c None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:10.273591 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8553d1-1cbd-4ee6-b664-c6bc543470d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:10.275353 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8553d1-1cbd-4ee6-b664-c6bc543470d8 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:10.276671 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8553d1-1cbd-4ee6-b664-c6bc543470d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:10.278304 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8553d1-1cbd-4ee6-b664-c6bc543470d8 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:10.279301 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8553d1-1cbd-4ee6-b664-c6bc543470d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:10.280014 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8553d1-1cbd-4ee6-b664-c6bc543470d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:10.282285 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8553d1-1cbd-4ee6-b664-c6bc543470d8 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:10.289868 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:10.300447 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:10.300632 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 394/780] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:33:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:10.309057 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8553d1-1cbd-4ee6-b664-c6bc543470d8 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:10.411374 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd2108a-75e3-4f50-922d-bb3f90641f44 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:10.457982 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8553d1-1cbd-4ee6-b664-c6bc543470d8 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:10.458207 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a358b7-9e26-4b96-ba99-4b86d21d36a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:10.458756 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a358b7-9e26-4b96-ba99-4b86d21d36a9 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:10.459044 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a358b7-9e26-4b96-ba99-4b86d21d36a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:10.459474 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a358b7-9e26-4b96-ba99-4b86d21d36a9 None admin] PATH_INFO: `/v3/services` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:10.460975 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a358b7-9e26-4b96-ba99-4b86d21d36a9 None admin] RBAC: Adding query filter params (name=network) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:10.461346 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a358b7-9e26-4b96-ba99-4b86d21d36a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:10.462713 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a358b7-9e26-4b96-ba99-4b86d21d36a9 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:10.469960 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 387/781] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Jul 6 13:33:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:10.475421 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a358b7-9e26-4b96-ba99-4b86d21d36a9 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:10.512117 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e298021-5a9e-4e7a-ba59-b38f1de85109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:10.512960 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e298021-5a9e-4e7a-ba59-b38f1de85109 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:10.513274 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e298021-5a9e-4e7a-ba59-b38f1de85109 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:10.513546 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e298021-5a9e-4e7a-ba59-b38f1de85109 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:10.524923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e298021-5a9e-4e7a-ba59-b38f1de85109 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:10.527029 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e298021-5a9e-4e7a-ba59-b38f1de85109 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:10.545890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-0e298021-5a9e-4e7a-ba59-b38f1de85109 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:10.545890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:10.545890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:10.545890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:10.545890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:10.545890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:10.545890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:10.545890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:10.545890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:10.545890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:10.545890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:10.545890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:10.545890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:10.545890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:10.545890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:10.546815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:10.546815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:10.546815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:10.546815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:10.546815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:10.546815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:10.546815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:10.546815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:10.546815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:10.546815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:10.546815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:10.546815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:10.546815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:10.546815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:10.546815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:10.546815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:10.546815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:10.547704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:10.547704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:10.547704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:10.547704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:10.547704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 395/782] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:10.553723 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e298021-5a9e-4e7a-ba59-b38f1de85109 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:10.631716 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a358b7-9e26-4b96-ba99-4b86d21d36a9 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:10.633320 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:10.669372 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d4047-f11c-4c23-882e-6a91eb406c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:10.670043 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d4047-f11c-4c23-882e-6a91eb406c5c None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:10.670325 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d4047-f11c-4c23-882e-6a91eb406c5c None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:10.670601 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d4047-f11c-4c23-882e-6a91eb406c5c None admin] PATH_INFO: `/v3/services` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:10.671316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d4047-f11c-4c23-882e-6a91eb406c5c None admin] RBAC: Adding query filter params (type=network) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:10.671624 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d4047-f11c-4c23-882e-6a91eb406c5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:10.672417 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:10.674270 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d4047-f11c-4c23-882e-6a91eb406c5c None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:10.680123 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 388/783] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Jul 6 13:33:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:10.686718 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d4047-f11c-4c23-882e-6a91eb406c5c None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:10.762928 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e298021-5a9e-4e7a-ba59-b38f1de85109 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:10.830224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd418fb-7694-41c7-a3c3-454395e22640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:10.830888 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd418fb-7694-41c7-a3c3-454395e22640 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:10.831169 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd418fb-7694-41c7-a3c3-454395e22640 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:10.831445 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd418fb-7694-41c7-a3c3-454395e22640 None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:10.832171 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd418fb-7694-41c7-a3c3-454395e22640 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:10.832476 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd418fb-7694-41c7-a3c3-454395e22640 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:10.833859 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd418fb-7694-41c7-a3c3-454395e22640 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:10.846639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 396/784] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:10.855501 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd418fb-7694-41c7-a3c3-454395e22640 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:10.886215 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d4047-f11c-4c23-882e-6a91eb406c5c None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:10.948489 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae8b51-bf88-45ec-afc0-c76cd140ebd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:10.949174 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae8b51-bf88-45ec-afc0-c76cd140ebd4 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:10.949458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae8b51-bf88-45ec-afc0-c76cd140ebd4 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:10.949728 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae8b51-bf88-45ec-afc0-c76cd140ebd4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:10.950517 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae8b51-bf88-45ec-afc0-c76cd140ebd4 None admin] RBAC: Adding query filter params (interface=public, service_id=07bcfe100cf149ce972c30316e05527c, region_id=RegionOne) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:10.950827 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae8b51-bf88-45ec-afc0-c76cd140ebd4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:10.952187 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae8b51-bf88-45ec-afc0-c76cd140ebd4 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:10.957522 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 389/785] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Jul 6 13:33:10 2022] GET /identity/v3/endpoints?service_id=07bcfe100cf149ce972c30316e05527c&interface=public®ion_id=RegionOne => generated 194 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:10.964981 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae8b51-bf88-45ec-afc0-c76cd140ebd4 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:11.005393 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:11.013928 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd418fb-7694-41c7-a3c3-454395e22640 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:11.051245 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d72f7e-e2ed-4174-9fe3-e170c3fa1f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:11.052179 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d72f7e-e2ed-4174-9fe3-e170c3fa1f16 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:11.052575 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d72f7e-e2ed-4174-9fe3-e170c3fa1f16 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:11.052930 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d72f7e-e2ed-4174-9fe3-e170c3fa1f16 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:11.057358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:11.061540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d72f7e-e2ed-4174-9fe3-e170c3fa1f16 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:11.063143 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d72f7e-e2ed-4174-9fe3-e170c3fa1f16 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:11.079215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-87d72f7e-e2ed-4174-9fe3-e170c3fa1f16 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 06 13:33:11.079215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:11.079215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:11.079215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:11.079215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:11.079215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:11.079215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:11.079215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:11.079215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:11.079215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:11.079215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:11.079215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:11.079215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:33:11.079215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:33:11.079215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:33:11.079215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:33:11.080404 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:11.080404 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:11.080404 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:33:11.080404 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:11.080404 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:33:11.080404 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:11.080404 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:11.080404 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:11.080404 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:11.080404 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:11.080404 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:11.080404 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:11.080404 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:11.080404 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:11.080404 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:11.080404 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:11.080404 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:11.080404 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:11.081611 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:11.081611 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:11.081611 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:33:11.081611 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:33:11.081611 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:33:11.081611 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:33:11.081611 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:33:11.081611 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:33:11.081611 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 06 13:33:11.081611 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:11.082971 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 397/786] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:33:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:33:11.089127 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d72f7e-e2ed-4174-9fe3-e170c3fa1f16 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:11.196956 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d72f7e-e2ed-4174-9fe3-e170c3fa1f16 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:11.217716 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae8b51-bf88-45ec-afc0-c76cd140ebd4 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:11.252940 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d14e5-fbef-415d-98b2-7c918877e084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:11.253640 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d14e5-fbef-415d-98b2-7c918877e084 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:11.253921 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d14e5-fbef-415d-98b2-7c918877e084 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:11.254193 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d14e5-fbef-415d-98b2-7c918877e084 None admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:11.255040 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24d14e5-fbef-415d-98b2-7c918877e084 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:11.261037 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24d14e5-fbef-415d-98b2-7c918877e084 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:11.262506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24d14e5-fbef-415d-98b2-7c918877e084 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:11.292890 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d9efe-6032-487b-a77e-26b8925dc811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:11.293512 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d9efe-6032-487b-a77e-26b8925dc811 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:11.293794 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d9efe-6032-487b-a77e-26b8925dc811 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:11.294061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d9efe-6032-487b-a77e-26b8925dc811 None admin] PATH_INFO: `/v3/services` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:11.298629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d9efe-6032-487b-a77e-26b8925dc811 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:11.300230 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 398/787] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Jul 6 13:33:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:11.300435 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d9efe-6032-487b-a77e-26b8925dc811 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:11.303666 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d14e5-fbef-415d-98b2-7c918877e084 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:11.315281 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 390/788] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Jul 6 13:33:10 2022] GET /identity/v3/services => generated 772 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:11.365436 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:11.442173 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d14e5-fbef-415d-98b2-7c918877e084 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:11.480973 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83633a-d24e-471a-b434-d0175054cfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:11.481803 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83633a-d24e-471a-b434-d0175054cfb2 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:11.482086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83633a-d24e-471a-b434-d0175054cfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:11.482368 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83633a-d24e-471a-b434-d0175054cfb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:11.493790 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f83633a-d24e-471a-b434-d0175054cfb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:11.495335 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f83633a-d24e-471a-b434-d0175054cfb2 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:11.514069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-1f83633a-d24e-471a-b434-d0175054cfb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:11.514069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:11.514069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:11.514069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:11.514069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:11.514069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:11.514069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:11.514069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:11.514069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:11.514069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:11.514069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:11.514069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:11.514069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:11.514069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:11.514069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:11.515094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:11.515094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:11.515094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:11.515094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:11.515094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:11.515094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:11.515094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:11.515094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:11.515094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:11.515094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:11.515094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:11.515094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:11.515094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:11.515094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:11.515094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:11.515094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:11.515094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:11.516011 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:11.516011 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:11.516011 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:11.516011 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:11.516011 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 399/789] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:11.522120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d9efe-6032-487b-a77e-26b8925dc811 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:11.629452 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:11.642613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d9efe-6032-487b-a77e-26b8925dc811 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:11.684603 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae313cd-3e8e-4e54-9504-15f4b5a44191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:11.685708 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae313cd-3e8e-4e54-9504-15f4b5a44191 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:11.686200 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae313cd-3e8e-4e54-9504-15f4b5a44191 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:11.686701 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae313cd-3e8e-4e54-9504-15f4b5a44191 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:11.687627 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae313cd-3e8e-4e54-9504-15f4b5a44191 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:11.688143 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae313cd-3e8e-4e54-9504-15f4b5a44191 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:11.689748 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae313cd-3e8e-4e54-9504-15f4b5a44191 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:11.706922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 391/790] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:11.712327 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83633a-d24e-471a-b434-d0175054cfb2 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:11.715261 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08609e4-7ec3-4329-bfb2-69ef64436b4a None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:11.715549 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08609e4-7ec3-4329-bfb2-69ef64436b4a None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:11.715824 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08609e4-7ec3-4329-bfb2-69ef64436b4a None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:11.717743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 392/791] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:33:11.723499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d73ea5-f88c-4b95-9f3b-cfa734ccfccc None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:11.723770 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d73ea5-f88c-4b95-9f3b-cfa734ccfccc None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:11.724046 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d73ea5-f88c-4b95-9f3b-cfa734ccfccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:11.744554 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-d6d73ea5-f88c-4b95-9f3b-cfa734ccfccc None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:11.746069 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-d6d73ea5-f88c-4b95-9f3b-cfa734ccfccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:11.829439 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 393/792] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 13:33:11.850105 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d73ea5-f88c-4b95-9f3b-cfa734ccfccc None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:11.891012 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:11.900606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83633a-d24e-471a-b434-d0175054cfb2 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:11.939585 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87fe051-133b-4f05-99a2-b589c4d1a2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:11.940500 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87fe051-133b-4f05-99a2-b589c4d1a2cf None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:11.941204 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87fe051-133b-4f05-99a2-b589c4d1a2cf None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:11.941612 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87fe051-133b-4f05-99a2-b589c4d1a2cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:11.949981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87fe051-133b-4f05-99a2-b589c4d1a2cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:11.951607 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87fe051-133b-4f05-99a2-b589c4d1a2cf None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:11.966302 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:11.967358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-f87fe051-133b-4f05-99a2-b589c4d1a2cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:11.967358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:11.967358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:11.967358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:11.967358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:11.967358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:11.967358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:11.967358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:11.967358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:11.967358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:11.967358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:11.967358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:11.967358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:33:11.967358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:33:11.967358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:33:11.968448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:33:11.968448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:11.968448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:11.968448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:11.968448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:11.968448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:11.968448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:11.968448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:11.968448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:11.968448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:11.968448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:11.968448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:11.968448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:11.968448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:11.968448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:11.968448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:11.968448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:11.969592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:33:11.969592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:33:11.969592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:33:11.969592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:33:11.969592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:33:11.969592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:33:11.969592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:11.969592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:11.977029 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 400/793] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:33:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:11.977029 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20db292-ea1e-4a61-a4bb-01ba4e855068 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:11.977029 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20db292-ea1e-4a61-a4bb-01ba4e855068 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:11.977029 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20db292-ea1e-4a61-a4bb-01ba4e855068 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:11.977029 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 401/794] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:11.977029 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20db292-ea1e-4a61-a4bb-01ba4e855068 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:12.007761 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d73ea5-f88c-4b95-9f3b-cfa734ccfccc None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:12.059323 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c22a237-7068-4970-8a69-48831cca736c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:12.060183 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22a237-7068-4970-8a69-48831cca736c None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:12.060433 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22a237-7068-4970-8a69-48831cca736c None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:12.060710 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22a237-7068-4970-8a69-48831cca736c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:12.072039 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c22a237-7068-4970-8a69-48831cca736c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:33:12.079900 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-0c22a237-7068-4970-8a69-48831cca736c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:33:12.079900 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:12.079900 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:12.079900 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:12.079900 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:12.079900 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:12.079900 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:12.079900 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:12.079900 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:12.079900 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:12.079900 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:12.079900 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:12.079900 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:33:12.079900 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:33:12.079900 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:33:12.079900 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:33:12.080905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:33:12.080905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:33:12.080905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:33:12.080905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:33:12.080905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:12.080905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:12.080905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:12.080905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:12.080905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:12.080905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:12.080905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:12.080905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:12.080905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:12.080905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:12.080905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:12.080905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:12.081781 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:12.081781 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:12.081781 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:12.081781 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:12.081781 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:33:12.081781 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:33:12.081781 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:33:12.081781 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:33:12.081781 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:33:12.081781 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:33:12.081781 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:33:12.081781 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:12.086063 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 394/795] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:33:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:12.087363 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366c302-e5fd-460d-87e5-58d4eb66171b None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:12.087650 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366c302-e5fd-460d-87e5-58d4eb66171b None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:12.087975 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366c302-e5fd-460d-87e5-58d4eb66171b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:12.105521 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-6366c302-e5fd-460d-87e5-58d4eb66171b None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:12.106706 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-6366c302-e5fd-460d-87e5-58d4eb66171b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:12.111791 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20db292-ea1e-4a61-a4bb-01ba4e855068 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:12.160112 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14ce6e-5b54-43c7-a889-16d5eb90aa7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:12.161197 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14ce6e-5b54-43c7-a889-16d5eb90aa7f None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:12.161773 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14ce6e-5b54-43c7-a889-16d5eb90aa7f None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:12.162312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14ce6e-5b54-43c7-a889-16d5eb90aa7f None admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:12.163377 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da14ce6e-5b54-43c7-a889-16d5eb90aa7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:12.163888 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da14ce6e-5b54-43c7-a889-16d5eb90aa7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:12.165910 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da14ce6e-5b54-43c7-a889-16d5eb90aa7f None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:12.185185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 402/796] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:33:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:12.187643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 395/797] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:12.190287 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14ce6e-5b54-43c7-a889-16d5eb90aa7f None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:12.193022 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366c302-e5fd-460d-87e5-58d4eb66171b None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:12.195663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:12.281336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:12.316664 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14ce6e-5b54-43c7-a889-16d5eb90aa7f None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:12.328271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366c302-e5fd-460d-87e5-58d4eb66171b None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:12.355984 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352d350-875a-4d2c-856c-07a1fa4d05d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:12.357022 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352d350-875a-4d2c-856c-07a1fa4d05d5 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:12.357544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352d350-875a-4d2c-856c-07a1fa4d05d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:12.358021 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352d350-875a-4d2c-856c-07a1fa4d05d5 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:12.358966 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2352d350-875a-4d2c-856c-07a1fa4d05d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:12.359471 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2352d350-875a-4d2c-856c-07a1fa4d05d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:12.361062 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2352d350-875a-4d2c-856c-07a1fa4d05d5 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:12.364635 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9acec31-3646-478a-a17b-915dbefc0d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:12.365554 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9acec31-3646-478a-a17b-915dbefc0d82 None admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:12.365832 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9acec31-3646-478a-a17b-915dbefc0d82 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:12.366101 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9acec31-3646-478a-a17b-915dbefc0d82 None admin] PATH_INFO: `/v3/projects/bfe6c3d8da2743be9609184db2506303/users/8c1bea05c37646b0a05fa8dda0367ef2/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:12.368111 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9acec31-3646-478a-a17b-915dbefc0d82 None admin] RBAC: Authorizing `identity:create_grant(project_id=bfe6c3d8da2743be9609184db2506303, user_id=8c1bea05c37646b0a05fa8dda0367ef2, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:12.369986 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9acec31-3646-478a-a17b-915dbefc0d82 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:12.372103 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 403/798] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:33:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:12.373204 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352d350-875a-4d2c-856c-07a1fa4d05d5 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:12.397241 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 396/799] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jul 6 13:33:12 2022] PUT /identity/v3/projects/bfe6c3d8da2743be9609184db2506303/users/8c1bea05c37646b0a05fa8dda0367ef2/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:33:12.398361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9acec31-3646-478a-a17b-915dbefc0d82 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:12.452656 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:12.533564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352d350-875a-4d2c-856c-07a1fa4d05d5 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:12.587041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e0b56-c5e1-470f-bd97-2750ffd7d372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:12.587849 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e0b56-c5e1-470f-bd97-2750ffd7d372 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:12.588188 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e0b56-c5e1-470f-bd97-2750ffd7d372 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:12.588501 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e0b56-c5e1-470f-bd97-2750ffd7d372 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:12.596330 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9acec31-3646-478a-a17b-915dbefc0d82 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:12.600126 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80e0b56-c5e1-470f-bd97-2750ffd7d372 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:33:12.608544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-a80e0b56-c5e1-470f-bd97-2750ffd7d372 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:33:12.608544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:12.608544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:12.608544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:12.608544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:12.608544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:12.608544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:12.608544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:12.608544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:12.608544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:12.608544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:12.608544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:12.608544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:33:12.608544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:33:12.608544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:33:12.608544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:33:12.609570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:33:12.609570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:33:12.609570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:33:12.609570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:33:12.609570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:12.609570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:12.609570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:12.609570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:12.609570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:12.609570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:12.609570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:12.609570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:12.609570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:12.609570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:12.609570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:12.609570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:12.610401 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:12.610401 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:12.610401 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:12.610401 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:12.610401 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:33:12.610401 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:33:12.610401 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:33:12.610401 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:33:12.610401 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:33:12.610401 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:33:12.610401 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:33:12.610401 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:12.610401 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 404/800] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jul 6 13:33:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:12.611575 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c48370-b918-451d-a8f1-3a08b7fa5593 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:12.611867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c48370-b918-451d-a8f1-3a08b7fa5593 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:12.612146 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c48370-b918-451d-a8f1-3a08b7fa5593 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:12.617785 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 405/801] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:12.623870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c48370-b918-451d-a8f1-3a08b7fa5593 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:12.637387 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:12.645735 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37a477-9d48-4bca-8296-51ae2ee9e871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:12.647304 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37a477-9d48-4bca-8296-51ae2ee9e871 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:12.647598 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37a477-9d48-4bca-8296-51ae2ee9e871 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:12.648027 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37a477-9d48-4bca-8296-51ae2ee9e871 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:12.659544 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37a477-9d48-4bca-8296-51ae2ee9e871 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:12.660991 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37a477-9d48-4bca-8296-51ae2ee9e871 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:12.680061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-bb37a477-9d48-4bca-8296-51ae2ee9e871 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:12.680061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:12.680061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:12.680061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:12.680061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:12.680061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:12.680061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:12.680061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:12.680061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:12.680061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:12.680061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:12.680061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:12.680061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:12.680061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:12.680061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:12.681065 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:12.681065 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:12.681065 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:12.681065 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:12.681065 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:12.681065 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:12.681065 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:12.681065 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:12.681065 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:12.681065 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:12.681065 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:12.681065 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:12.681065 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:12.681065 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:12.681065 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:12.681065 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:12.681065 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:12.682749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:12.682749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:12.682749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:12.682749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:12.682749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 397/802] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:12.682749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba80617-7726-497c-90dd-0596f186162e None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:12.683126 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba80617-7726-497c-90dd-0596f186162e None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:12.683195 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba80617-7726-497c-90dd-0596f186162e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:12.700512 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-2ba80617-7726-497c-90dd-0596f186162e None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:12.701743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-2ba80617-7726-497c-90dd-0596f186162e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:12.758235 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c48370-b918-451d-a8f1-3a08b7fa5593 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:12.790552 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:12.791570 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 398/803] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:12.792633 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba80617-7726-497c-90dd-0596f186162e None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:12.829090 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2d24c-05b2-4445-9a83-84e006588d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:12.829765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2d24c-05b2-4445-9a83-84e006588d5b None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:12.830053 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2d24c-05b2-4445-9a83-84e006588d5b None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:12.830332 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2d24c-05b2-4445-9a83-84e006588d5b None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:12.831093 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff2d24c-05b2-4445-9a83-84e006588d5b None admin] RBAC: Adding query filter params (name=service) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:12.831403 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff2d24c-05b2-4445-9a83-84e006588d5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:12.832876 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff2d24c-05b2-4445-9a83-84e006588d5b None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:12.842351 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 406/804] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jul 6 13:33:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:12.843181 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2d24c-05b2-4445-9a83-84e006588d5b None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:12.932859 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba80617-7726-497c-90dd-0596f186162e None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:12.952271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:12.960397 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2d24c-05b2-4445-9a83-84e006588d5b None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:12.977867 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745dcb4-ee43-4bb3-adb1-8d21edb28ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:12.978502 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745dcb4-ee43-4bb3-adb1-8d21edb28ee6 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:12.979007 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745dcb4-ee43-4bb3-adb1-8d21edb28ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:12.979293 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745dcb4-ee43-4bb3-adb1-8d21edb28ee6 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:12.980029 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d745dcb4-ee43-4bb3-adb1-8d21edb28ee6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:12.980341 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d745dcb4-ee43-4bb3-adb1-8d21edb28ee6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:12.981748 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d745dcb4-ee43-4bb3-adb1-8d21edb28ee6 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:12.998381 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 399/805] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:12.998381 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745dcb4-ee43-4bb3-adb1-8d21edb28ee6 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:13.014062 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd92993-0161-4d4a-9acf-a2e69e7f4180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:13.015370 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd92993-0161-4d4a-9acf-a2e69e7f4180 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:13.016242 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd92993-0161-4d4a-9acf-a2e69e7f4180 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:13.016669 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd92993-0161-4d4a-9acf-a2e69e7f4180 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:13.017756 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd92993-0161-4d4a-9acf-a2e69e7f4180 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:13.019891 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd92993-0161-4d4a-9acf-a2e69e7f4180 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:13.037494 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-fbd92993-0161-4d4a-9acf-a2e69e7f4180 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 06 13:33:13.037494 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:13.037494 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 13:33:13.037494 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 13:33:13.037494 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 13:33:13.037494 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 13:33:13.037494 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 13:33:13.037494 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:13.037494 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 06 13:33:13.037494 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:13.037494 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 13:33:13.037494 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:13.037494 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:13.037494 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:13.037494 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:13.037494 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:13.038630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:13.038630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:13.038630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:13.038630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:13.038630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:13.038630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:13.038630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:13.038630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 13:33:13.038630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 13:33:13.038630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 13:33:13.038630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 13:33:13.038630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:13.038630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:13.038630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:13.038630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:13.038630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:13.039639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:13.039639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:13.039639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:13.039639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:13.039639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:13.039639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:13.039639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:13.039639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:13.039639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:13.039639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:13.039639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:13.039639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 13:33:13.039639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:13.039639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 06 13:33:13.039639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:13.047063 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 407/806] 10.222.0.20 () {56 vars in 1129 bytes} [Wed Jul 6 13:33:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:13.047063 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd92993-0161-4d4a-9acf-a2e69e7f4180 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:13.103824 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:13.132326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745dcb4-ee43-4bb3-adb1-8d21edb28ee6 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:13.158753 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd92993-0161-4d4a-9acf-a2e69e7f4180 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:13.182416 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a93b03-a9c6-4c2a-a818-5bc63ed7fd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:13.184227 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a93b03-a9c6-4c2a-a818-5bc63ed7fd4a None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:13.184981 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a93b03-a9c6-4c2a-a818-5bc63ed7fd4a None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:13.185478 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a93b03-a9c6-4c2a-a818-5bc63ed7fd4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:13.194820 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9828063-2143-4b59-8201-dbd4c2cf6050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:13.195645 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9828063-2143-4b59-8201-dbd4c2cf6050 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:13.196242 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9828063-2143-4b59-8201-dbd4c2cf6050 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:13.196242 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9828063-2143-4b59-8201-dbd4c2cf6050 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:13.201990 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a93b03-a9c6-4c2a-a818-5bc63ed7fd4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:13.203861 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a93b03-a9c6-4c2a-a818-5bc63ed7fd4a None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:13.204869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9828063-2143-4b59-8201-dbd4c2cf6050 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:13.206403 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9828063-2143-4b59-8201-dbd4c2cf6050 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:13.223311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-c9828063-2143-4b59-8201-dbd4c2cf6050 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 06 13:33:13.223311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:13.223311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:13.223311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:13.223311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:13.223311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:13.223311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:13.223311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:13.223311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:13.223311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:13.223311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:13.223311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:13.223311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:33:13.223311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:33:13.223311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:33:13.223311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:33:13.224321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:13.224321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:13.224321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:33:13.224321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:13.224321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:33:13.224321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:13.224321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:13.224321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:13.224321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:13.224321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:13.224321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:13.224321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:13.224321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:13.224321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:13.224321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:13.224321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:13.224321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:13.224321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:13.225681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:13.225681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:13.225681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:33:13.225681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:33:13.225681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:33:13.225681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:33:13.225681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:33:13.225681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:33:13.225681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 06 13:33:13.225681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:13.227357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-02a93b03-a9c6-4c2a-a818-5bc63ed7fd4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:13.227357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:13.227357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:13.227357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:13.227357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:13.227357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:13.227357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:13.227357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:13.227357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:13.227357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:13.227357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:13.227357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:13.227357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:13.227357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:13.227357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:13.228211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:13.228211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:13.228211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:13.228211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:13.228211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:13.228211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:13.228211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:13.228211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:13.228211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:13.228211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:13.228211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:13.228211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:13.228211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:13.228211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:13.228211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:13.228211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:13.228211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:13.229669 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:13.229669 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:13.229669 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:13.229669 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:13.229990 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 408/807] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Jul 6 13:33:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:13.230575 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9828063-2143-4b59-8201-dbd4c2cf6050 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:13.232424 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 400/808] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:13.233873 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a93b03-a9c6-4c2a-a818-5bc63ed7fd4a None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:13.268221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:13.354021 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a93b03-a9c6-4c2a-a818-5bc63ed7fd4a None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:13.361257 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9828063-2143-4b59-8201-dbd4c2cf6050 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:13.374654 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:13.392160 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9066cf27-39e1-4f17-89bb-08b6cf16dd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:13.393304 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066cf27-39e1-4f17-89bb-08b6cf16dd69 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:13.393781 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066cf27-39e1-4f17-89bb-08b6cf16dd69 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:13.394615 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066cf27-39e1-4f17-89bb-08b6cf16dd69 None admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:13.395866 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9066cf27-39e1-4f17-89bb-08b6cf16dd69 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:13.396354 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9066cf27-39e1-4f17-89bb-08b6cf16dd69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:13.398867 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9066cf27-39e1-4f17-89bb-08b6cf16dd69 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:13.407023 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb181df-29ef-4f55-aa5e-060821d1686c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:13.407815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb181df-29ef-4f55-aa5e-060821d1686c None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:13.408220 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb181df-29ef-4f55-aa5e-060821d1686c None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:13.408601 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb181df-29ef-4f55-aa5e-060821d1686c None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:13.409464 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb181df-29ef-4f55-aa5e-060821d1686c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:13.409875 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb181df-29ef-4f55-aa5e-060821d1686c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:13.411920 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb181df-29ef-4f55-aa5e-060821d1686c None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:13.421105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 409/809] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Jul 6 13:33:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:13.421105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb181df-29ef-4f55-aa5e-060821d1686c None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:13.431191 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 401/810] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Jul 6 13:33:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:13.432299 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9066cf27-39e1-4f17-89bb-08b6cf16dd69 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:13.536610 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb181df-29ef-4f55-aa5e-060821d1686c None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:13.540457 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:13.564855 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9066cf27-39e1-4f17-89bb-08b6cf16dd69 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:13.576821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c500ee-11af-4061-bb5a-07ca888a5ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:13.577525 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c500ee-11af-4061-bb5a-07ca888a5ff0 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:13.577812 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c500ee-11af-4061-bb5a-07ca888a5ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:13.578124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c500ee-11af-4061-bb5a-07ca888a5ff0 None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:13.578992 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c500ee-11af-4061-bb5a-07ca888a5ff0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:13.579307 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c500ee-11af-4061-bb5a-07ca888a5ff0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:13.580828 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c500ee-11af-4061-bb5a-07ca888a5ff0 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:13.599240 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 410/811] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:13.600117 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c500ee-11af-4061-bb5a-07ca888a5ff0 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:13.612971 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02eb05-f4ce-4a6d-b9b9-48e2d67bbe03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:13.614324 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02eb05-f4ce-4a6d-b9b9-48e2d67bbe03 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:13.615029 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02eb05-f4ce-4a6d-b9b9-48e2d67bbe03 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:13.615464 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02eb05-f4ce-4a6d-b9b9-48e2d67bbe03 None admin] PATH_INFO: `/v3/services` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:13.616564 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02eb05-f4ce-4a6d-b9b9-48e2d67bbe03 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:13.617513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02eb05-f4ce-4a6d-b9b9-48e2d67bbe03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:13.619371 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02eb05-f4ce-4a6d-b9b9-48e2d67bbe03 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:13.631110 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 402/812] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Jul 6 13:33:13 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:13.631110 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02eb05-f4ce-4a6d-b9b9-48e2d67bbe03 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:13.642876 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:13.716941 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c500ee-11af-4061-bb5a-07ca888a5ff0 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:13.739265 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02eb05-f4ce-4a6d-b9b9-48e2d67bbe03 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:13.754560 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13409a43-245c-4552-8b73-33537c5c3c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:13.755322 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13409a43-245c-4552-8b73-33537c5c3c13 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:13.755606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13409a43-245c-4552-8b73-33537c5c3c13 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:13.755995 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13409a43-245c-4552-8b73-33537c5c3c13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:13.772685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a7e6b-d221-44d3-875f-d5dd6aee71f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:13.773523 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a7e6b-d221-44d3-875f-d5dd6aee71f4 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:13.773811 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a7e6b-d221-44d3-875f-d5dd6aee71f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:13.774084 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a7e6b-d221-44d3-875f-d5dd6aee71f4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:13.781828 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a7e6b-d221-44d3-875f-d5dd6aee71f4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:13.783259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a7e6b-d221-44d3-875f-d5dd6aee71f4 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:13.798984 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-292a7e6b-d221-44d3-875f-d5dd6aee71f4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 13:33:13.798984 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:13.798984 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:13.798984 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:13.798984 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:13.798984 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:13.798984 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:13.798984 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:13.798984 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:13.798984 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:13.798984 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:13.798984 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:13.798984 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:33:13.798984 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:33:13.798984 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:33:13.798984 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:33:13.799918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:13.799918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:13.799918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:33:13.799918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:13.799918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:33:13.799918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:13.799918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:13.799918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:13.799918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:13.799918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:13.799918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:13.799918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:13.799918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:13.799918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:13.799918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:13.799918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:13.799918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:13.799918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:13.800901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:13.800901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:13.800901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:33:13.800901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:33:13.800901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:33:13.800901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:33:13.800901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:33:13.800901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:33:13.800901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 13:33:13.800901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:13.800901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 403/813] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Jul 6 13:33:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:13.800901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a7e6b-d221-44d3-875f-d5dd6aee71f4 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:13.813205 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:13.836490 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13409a43-245c-4552-8b73-33537c5c3c13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:13.838744 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13409a43-245c-4552-8b73-33537c5c3c13 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:13.873829 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:13.906913 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a7e6b-d221-44d3-875f-d5dd6aee71f4 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:13.943890 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c666a-97d0-40eb-a6b1-a109e5d2c464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:13.944542 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c666a-97d0-40eb-a6b1-a109e5d2c464 None admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:13.944824 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c666a-97d0-40eb-a6b1-a109e5d2c464 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:13.945138 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c666a-97d0-40eb-a6b1-a109e5d2c464 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:13.945874 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c666a-97d0-40eb-a6b1-a109e5d2c464 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:13.947796 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c666a-97d0-40eb-a6b1-a109e5d2c464 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:13.949883 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 411/814] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:33:13 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 13:33:13.951289 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482ad0b-c65d-4866-9d35-302691d9fdc8 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:13.951580 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482ad0b-c65d-4866-9d35-302691d9fdc8 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:13.951987 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482ad0b-c65d-4866-9d35-302691d9fdc8 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:13.957066 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 412/815] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:13.957066 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482ad0b-c65d-4866-9d35-302691d9fdc8 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:13.981041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 404/816] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Jul 6 13:33:13 2022] POST /identity/v3/endpoints => generated 346 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:33:13.981041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c666a-97d0-40eb-a6b1-a109e5d2c464 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:14.053249 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:14.098278 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482ad0b-c65d-4866-9d35-302691d9fdc8 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:14.104492 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c666a-97d0-40eb-a6b1-a109e5d2c464 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:14.131666 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:14.140244 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8ca83-daf0-42ed-bbdb-681c31c0b31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:14.141311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8ca83-daf0-42ed-bbdb-681c31c0b31d None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:14.141797 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8ca83-daf0-42ed-bbdb-681c31c0b31d None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:14.142269 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8ca83-daf0-42ed-bbdb-681c31c0b31d None admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:14.143352 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b8ca83-daf0-42ed-bbdb-681c31c0b31d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:14.143867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b8ca83-daf0-42ed-bbdb-681c31c0b31d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:14.145559 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b8ca83-daf0-42ed-bbdb-681c31c0b31d None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:14.145941 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6ea31-c4c4-44c0-ae34-c56ee39a0c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:14.146750 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6ea31-c4c4-44c0-ae34-c56ee39a0c06 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:14.147088 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6ea31-c4c4-44c0-ae34-c56ee39a0c06 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:14.147367 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6ea31-c4c4-44c0-ae34-c56ee39a0c06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:14.160468 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6ea31-c4c4-44c0-ae34-c56ee39a0c06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:14.162007 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6ea31-c4c4-44c0-ae34-c56ee39a0c06 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:14.169092 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 413/817] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Jul 6 13:33:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:14.170504 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7e2c4-4984-4e03-9947-1e0aa9339b19 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:14.170792 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7e2c4-4984-4e03-9947-1e0aa9339b19 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:14.171067 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7e2c4-4984-4e03-9947-1e0aa9339b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:14.181516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-5cb6ea31-c4c4-44c0-ae34-c56ee39a0c06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:14.181516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:14.181516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:14.181516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:14.181516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:14.181516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:14.181516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:14.181516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:14.181516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:14.181516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:14.181516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:14.181516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:14.181516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:14.181516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:14.181516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:14.182953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:14.182953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:14.182953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:14.182953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:14.182953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:14.182953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:14.182953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:14.182953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:14.182953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:14.182953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:14.182953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:14.182953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:14.182953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:14.182953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:14.182953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:14.182953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:14.182953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:14.183834 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:14.183834 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:14.183834 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:14.183834 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:14.183834 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 405/818] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:14.184120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6ea31-c4c4-44c0-ae34-c56ee39a0c06 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:14.190471 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-53c7e2c4-4984-4e03-9947-1e0aa9339b19 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:14.191414 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-53c7e2c4-4984-4e03-9947-1e0aa9339b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:14.265957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 414/819] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:14.266915 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c7e2c4-4984-4e03-9947-1e0aa9339b19 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:14.336804 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6ea31-c4c4-44c0-ae34-c56ee39a0c06 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:14.354952 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:14.378747 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5690e8-886c-4646-8d79-bbedf3f2c656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:14.379643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5690e8-886c-4646-8d79-bbedf3f2c656 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:14.379934 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5690e8-886c-4646-8d79-bbedf3f2c656 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:14.380213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5690e8-886c-4646-8d79-bbedf3f2c656 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:14.392053 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5690e8-886c-4646-8d79-bbedf3f2c656 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:14.393641 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5690e8-886c-4646-8d79-bbedf3f2c656 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:14.395644 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c7e2c4-4984-4e03-9947-1e0aa9339b19 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:14.412386 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-6f5690e8-886c-4646-8d79-bbedf3f2c656 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:14.412386 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:14.412386 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:14.412386 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:14.412386 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:14.412386 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:14.412386 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:14.412386 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:14.412386 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:14.412386 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:14.412386 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:14.412386 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:14.412386 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:14.412386 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:14.412386 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:14.413340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:14.413340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:14.413340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:14.413340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:14.413340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:14.413340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:14.413340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:14.413340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:14.413340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:14.413340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:14.413340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:14.413340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:14.413340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:14.413340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:14.413340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:14.413340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:14.413340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:14.414305 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:14.414305 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:14.414305 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:14.414305 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:14.414305 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 406/820] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:14.415015 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5690e8-886c-4646-8d79-bbedf3f2c656 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:14.418913 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:14.439149 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f7c26-64d1-4061-a45f-66bcdd983fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:14.439866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f7c26-64d1-4061-a45f-66bcdd983fcf None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:14.440145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f7c26-64d1-4061-a45f-66bcdd983fcf None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:14.440463 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f7c26-64d1-4061-a45f-66bcdd983fcf None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:14.441223 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2f7c26-64d1-4061-a45f-66bcdd983fcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:14.441527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2f7c26-64d1-4061-a45f-66bcdd983fcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:14.442954 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2f7c26-64d1-4061-a45f-66bcdd983fcf None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:14.455437 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 415/821] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:14.456329 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f7c26-64d1-4061-a45f-66bcdd983fcf None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:14.534564 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5690e8-886c-4646-8d79-bbedf3f2c656 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:14.565475 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f7c26-64d1-4061-a45f-66bcdd983fcf None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:14.584419 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247b844-3675-44f7-b464-9a05cb90ad88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:14.585252 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247b844-3675-44f7-b464-9a05cb90ad88 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:14.586060 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247b844-3675-44f7-b464-9a05cb90ad88 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:14.586344 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247b844-3675-44f7-b464-9a05cb90ad88 None admin] PATH_INFO: `/v3/services/network` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:14.587345 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9247b844-3675-44f7-b464-9a05cb90ad88 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:14.589473 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9247b844-3675-44f7-b464-9a05cb90ad88 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:14.600424 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad90393-9635-4013-9a80-b0bd4f005de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:14.601114 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad90393-9635-4013-9a80-b0bd4f005de3 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:14.601399 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad90393-9635-4013-9a80-b0bd4f005de3 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:14.601669 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad90393-9635-4013-9a80-b0bd4f005de3 None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:14.602388 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad90393-9635-4013-9a80-b0bd4f005de3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:14.602688 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad90393-9635-4013-9a80-b0bd4f005de3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:14.604467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-9247b844-3675-44f7-b464-9a05cb90ad88 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 06 13:33:14.604467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:14.604467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 13:33:14.604467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 13:33:14.604467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 13:33:14.604467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 13:33:14.604467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 13:33:14.604467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:14.604467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 06 13:33:14.604467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:14.604467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 13:33:14.604467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:14.604467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:14.604467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:14.604467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:14.604467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:14.604467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:14.605744 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:14.605744 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:14.605744 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:14.605744 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:14.605744 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:14.605744 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:14.605744 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 13:33:14.605744 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 13:33:14.605744 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 13:33:14.605744 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 13:33:14.605744 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:14.605744 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:14.605744 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:14.605744 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:14.605744 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:14.605744 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:14.606629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:14.606629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:14.606629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:14.606629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:14.606629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:14.606629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:14.606629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:14.606629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:14.606629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:14.606629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:14.606629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 13:33:14.606629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:14.606629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 06 13:33:14.606629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:14.607383 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad90393-9635-4013-9a80-b0bd4f005de3 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:14.607457 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 407/822] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:33:14 2022] GET /identity/v3/services/network => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:14.607457 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247b844-3675-44f7-b464-9a05cb90ad88 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:14.621725 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 416/823] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:14.623438 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18490558-c987-4c37-b067-0918ea08c9df None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:14.623795 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18490558-c987-4c37-b067-0918ea08c9df None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:14.624150 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18490558-c987-4c37-b067-0918ea08c9df None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:14.625665 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 417/824] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:14.630029 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18490558-c987-4c37-b067-0918ea08c9df None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:14.642174 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:14.679359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:14.717633 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247b844-3675-44f7-b464-9a05cb90ad88 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:14.746758 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18490558-c987-4c37-b067-0918ea08c9df None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:14.754699 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55115798-3321-45b4-b5d0-46e90a8ce98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:14.755298 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55115798-3321-45b4-b5d0-46e90a8ce98b None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:14.755586 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55115798-3321-45b4-b5d0-46e90a8ce98b None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:14.755865 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55115798-3321-45b4-b5d0-46e90a8ce98b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:14.765450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55115798-3321-45b4-b5d0-46e90a8ce98b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:14.766897 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55115798-3321-45b4-b5d0-46e90a8ce98b None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:14.781588 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-55115798-3321-45b4-b5d0-46e90a8ce98b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:14.781588 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:14.781588 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:14.781588 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:14.781588 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:14.781588 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:14.781588 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:14.781588 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:14.781588 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:14.781588 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:14.781588 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:14.781588 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:14.781588 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:33:14.781588 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:33:14.781588 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:33:14.782506 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:33:14.782506 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:14.782506 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:14.782506 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:14.782506 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:14.782506 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:14.782506 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:14.782506 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:14.782506 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:14.782506 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:14.782506 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:14.782506 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:14.782506 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:14.782506 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:14.782506 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:14.782506 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:14.782506 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:14.788353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:33:14.788353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:33:14.788353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:33:14.788353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:33:14.788353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:33:14.788353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:33:14.788353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:14.788353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:14.788353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 408/825] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:33:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:14.788353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55115798-3321-45b4-b5d0-46e90a8ce98b None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:14.792829 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269bbbcc-4c81-43c3-ad94-a887c84f0808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:14.793548 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bbbcc-4c81-43c3-ad94-a887c84f0808 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:14.793828 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bbbcc-4c81-43c3-ad94-a887c84f0808 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:14.794106 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bbbcc-4c81-43c3-ad94-a887c84f0808 None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:14.794816 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269bbbcc-4c81-43c3-ad94-a887c84f0808 None admin] RBAC: Adding query filter params (name=network) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:14.795119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269bbbcc-4c81-43c3-ad94-a887c84f0808 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:14.796488 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269bbbcc-4c81-43c3-ad94-a887c84f0808 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:14.801480 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 418/826] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Jul 6 13:33:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:14.807216 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003ae64-d1e1-44f3-a5f6-ba996a796762 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:14.807755 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003ae64-d1e1-44f3-a5f6-ba996a796762 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:14.808236 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003ae64-d1e1-44f3-a5f6-ba996a796762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:14.828093 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-6003ae64-d1e1-44f3-a5f6-ba996a796762 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:14.829695 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-6003ae64-d1e1-44f3-a5f6-ba996a796762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:14.887482 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:14.898006 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55115798-3321-45b4-b5d0-46e90a8ce98b None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:14.913053 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 419/827] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:14.913053 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003ae64-d1e1-44f3-a5f6-ba996a796762 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:14.941457 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd9011-95aa-432f-95da-d32891477e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:14.942404 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd9011-95aa-432f-95da-d32891477e24 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:14.942783 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd9011-95aa-432f-95da-d32891477e24 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:14.943187 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd9011-95aa-432f-95da-d32891477e24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:14.951407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dd9011-95aa-432f-95da-d32891477e24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:14.952965 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dd9011-95aa-432f-95da-d32891477e24 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:14.954454 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:14.967741 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-60dd9011-95aa-432f-95da-d32891477e24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:14.967741 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:14.967741 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:14.967741 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:14.967741 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:14.967741 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:14.967741 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:14.967741 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:14.967741 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:14.967741 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:14.967741 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:14.967741 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:14.967741 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:33:14.967741 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:33:14.967741 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:33:14.968808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:33:14.968808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:14.968808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:14.968808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:14.968808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:14.968808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:14.968808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:14.968808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:14.968808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:14.968808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:14.968808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:14.968808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:14.968808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:14.968808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:14.968808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:14.968808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:14.968808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:14.969904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:33:14.969904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:33:14.969904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:33:14.969904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:33:14.969904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:33:14.969904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:33:14.969904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:14.969904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:14.975534 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 409/828] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:33:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:14.975534 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd9011-95aa-432f-95da-d32891477e24 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:15.034016 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003ae64-d1e1-44f3-a5f6-ba996a796762 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:15.068617 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe5282-8e79-41dc-a44d-f8819bbd907e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:15.069309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe5282-8e79-41dc-a44d-f8819bbd907e None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:15.069594 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe5282-8e79-41dc-a44d-f8819bbd907e None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:15.069880 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe5282-8e79-41dc-a44d-f8819bbd907e None admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:15.070666 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe5282-8e79-41dc-a44d-f8819bbd907e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:15.070974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe5282-8e79-41dc-a44d-f8819bbd907e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:15.072975 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe5282-8e79-41dc-a44d-f8819bbd907e None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:15.079434 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd9011-95aa-432f-95da-d32891477e24 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:15.089431 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 420/829] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:33:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:15.089431 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe5282-8e79-41dc-a44d-f8819bbd907e None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:15.116981 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8025dee3-fa0a-442c-8696-2033d238122c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:15.117763 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025dee3-fa0a-442c-8696-2033d238122c None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:15.118061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025dee3-fa0a-442c-8696-2033d238122c None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:15.118343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025dee3-fa0a-442c-8696-2033d238122c None admin] PATH_INFO: `/v3/services` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:15.119563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8025dee3-fa0a-442c-8696-2033d238122c None admin] RBAC: Adding query filter params (type=network) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:15.119881 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8025dee3-fa0a-442c-8696-2033d238122c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:15.121269 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8025dee3-fa0a-442c-8696-2033d238122c None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:15.130468 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 410/830] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Jul 6 13:33:14 2022] GET /identity/v3/services?type=network => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:15.130468 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8025dee3-fa0a-442c-8696-2033d238122c None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:15.144750 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:15.196795 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:15.207722 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe5282-8e79-41dc-a44d-f8819bbd907e None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:15.248077 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8025dee3-fa0a-442c-8696-2033d238122c None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:15.250062 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340cac2f-133c-4d61-aa30-a14263ae3ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:15.250858 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340cac2f-133c-4d61-aa30-a14263ae3ccf None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:15.251141 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340cac2f-133c-4d61-aa30-a14263ae3ccf None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:15.251419 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340cac2f-133c-4d61-aa30-a14263ae3ccf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:15.261920 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340cac2f-133c-4d61-aa30-a14263ae3ccf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:33:15.269431 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-340cac2f-133c-4d61-aa30-a14263ae3ccf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:33:15.269431 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:15.269431 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:15.269431 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:15.269431 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:15.269431 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:15.269431 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:15.269431 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:15.269431 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:15.269431 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:15.269431 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:15.269431 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:15.269431 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:33:15.269431 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:33:15.269431 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:33:15.269431 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:33:15.270364 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:33:15.270364 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:33:15.270364 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:33:15.270364 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:33:15.270364 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:15.270364 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:15.270364 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:15.270364 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:15.270364 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:15.270364 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:15.270364 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:15.270364 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:15.270364 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:15.270364 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:15.270364 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:15.270364 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:15.271225 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:15.271225 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:15.271225 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:15.271225 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:15.271225 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:33:15.271225 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:33:15.271225 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:33:15.271225 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:33:15.271225 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:33:15.271225 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:33:15.271225 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:33:15.271225 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:15.271225 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 421/831] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:33:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:15.271923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340cac2f-133c-4d61-aa30-a14263ae3ccf None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:15.292787 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45157e17-684f-4dfe-bb4e-4f7716682afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:15.293790 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45157e17-684f-4dfe-bb4e-4f7716682afc None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:15.294246 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45157e17-684f-4dfe-bb4e-4f7716682afc None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:15.294721 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45157e17-684f-4dfe-bb4e-4f7716682afc None admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:15.295843 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45157e17-684f-4dfe-bb4e-4f7716682afc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:15.296134 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45157e17-684f-4dfe-bb4e-4f7716682afc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:15.297486 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45157e17-684f-4dfe-bb4e-4f7716682afc None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:15.311583 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 411/832] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:33:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:15.312528 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45157e17-684f-4dfe-bb4e-4f7716682afc None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:15.391942 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340cac2f-133c-4d61-aa30-a14263ae3ccf None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:15.395924 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:15.426736 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7dd119-7e7e-40ef-90b1-4d9ab44af201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:15.427388 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7dd119-7e7e-40ef-90b1-4d9ab44af201 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:15.427668 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7dd119-7e7e-40ef-90b1-4d9ab44af201 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:15.427944 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7dd119-7e7e-40ef-90b1-4d9ab44af201 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:15.428763 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7dd119-7e7e-40ef-90b1-4d9ab44af201 None admin] RBAC: Adding query filter params (role.id=795daa2c78354fd48e4a741e1182cc98, scope.project.id=bfe6c3d8da2743be9609184db2506303, user.id=6647eb1e8a1544ef9d3a9806f2d1fc51) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:15.429173 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7dd119-7e7e-40ef-90b1-4d9ab44af201 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:15.429998 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45157e17-684f-4dfe-bb4e-4f7716682afc None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:15.430586 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7dd119-7e7e-40ef-90b1-4d9ab44af201 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:15.436637 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 422/833] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:33:15 2022] GET /identity/v3/role_assignments?user.id=6647eb1e8a1544ef9d3a9806f2d1fc51&scope.project.id=bfe6c3d8da2743be9609184db2506303&role.id=795daa2c78354fd48e4a741e1182cc98 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:15.437748 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7dd119-7e7e-40ef-90b1-4d9ab44af201 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:15.467570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:15.471084 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95df07-a6c4-40cd-9777-9372e7f2ac9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:15.472046 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95df07-a6c4-40cd-9777-9372e7f2ac9e None admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:15.472528 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95df07-a6c4-40cd-9777-9372e7f2ac9e None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:15.472780 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95df07-a6c4-40cd-9777-9372e7f2ac9e None admin] PATH_INFO: `/v3/endpoints` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:15.473593 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e95df07-a6c4-40cd-9777-9372e7f2ac9e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:15.475083 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e95df07-a6c4-40cd-9777-9372e7f2ac9e None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:15.497511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 412/834] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Jul 6 13:33:15 2022] POST /identity/v3/endpoints => generated 327 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:33:15.506219 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95df07-a6c4-40cd-9777-9372e7f2ac9e None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:15.603248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7dd119-7e7e-40ef-90b1-4d9ab44af201 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:15.664403 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95df07-a6c4-40cd-9777-9372e7f2ac9e None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:15.682424 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab174c-d37d-4fe7-b75f-47b40f76d075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:15.683484 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab174c-d37d-4fe7-b75f-47b40f76d075 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:15.685369 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab174c-d37d-4fe7-b75f-47b40f76d075 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:15.685656 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab174c-d37d-4fe7-b75f-47b40f76d075 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:15.686447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab174c-d37d-4fe7-b75f-47b40f76d075 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:15.686760 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab174c-d37d-4fe7-b75f-47b40f76d075 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:15.688166 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab174c-d37d-4fe7-b75f-47b40f76d075 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:15.697924 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 423/835] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:33:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:15.703480 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:15.703658 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab174c-d37d-4fe7-b75f-47b40f76d075 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:15.718630 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090b7dd-5959-4cf6-9fad-1a286f94f7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:15.720090 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090b7dd-5959-4cf6-9fad-1a286f94f7dd None admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:15.720849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090b7dd-5959-4cf6-9fad-1a286f94f7dd None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:15.721216 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090b7dd-5959-4cf6-9fad-1a286f94f7dd None admin] PATH_INFO: `/v3/projects/bfe6c3d8da2743be9609184db2506303/users/c4535134181e489fb532f617264d5855/roles/a6853d485f7942259b3646de3f386852` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:15.743910 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7090b7dd-5959-4cf6-9fad-1a286f94f7dd None admin] RBAC: Authorizing `identity:create_grant(project_id=bfe6c3d8da2743be9609184db2506303, user_id=c4535134181e489fb532f617264d5855, role_id=a6853d485f7942259b3646de3f386852)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:15.746634 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7090b7dd-5959-4cf6-9fad-1a286f94f7dd None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:15.771376 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 413/836] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jul 6 13:33:15 2022] PUT /identity/v3/projects/bfe6c3d8da2743be9609184db2506303/users/c4535134181e489fb532f617264d5855/roles/a6853d485f7942259b3646de3f386852 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:33:15.810715 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:15.841352 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab174c-d37d-4fe7-b75f-47b40f76d075 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:15.881418 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f8577-aec4-4664-9ddc-1e9b9d077766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:15.882490 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f8577-aec4-4664-9ddc-1e9b9d077766 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:15.882949 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f8577-aec4-4664-9ddc-1e9b9d077766 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:15.883371 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f8577-aec4-4664-9ddc-1e9b9d077766 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:15.895925 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f8577-aec4-4664-9ddc-1e9b9d077766 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:15.897658 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f8577-aec4-4664-9ddc-1e9b9d077766 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:15.920824 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-308f8577-aec4-4664-9ddc-1e9b9d077766 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:15.920824 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:15.920824 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:15.920824 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:15.920824 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:15.920824 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:15.920824 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:15.920824 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:15.920824 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:15.920824 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:15.920824 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:15.920824 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:15.920824 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:15.920824 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:15.920824 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:15.921989 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:15.921989 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:15.921989 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:15.921989 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:15.921989 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:15.921989 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:15.921989 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:15.921989 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:15.921989 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:15.921989 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:15.921989 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:15.921989 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:15.921989 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:15.921989 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:15.921989 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:15.921989 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:15.921989 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:15.923081 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:15.923081 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:15.923081 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:15.923081 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:15.925399 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 424/837] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:15.931067 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090b7dd-5959-4cf6-9fad-1a286f94f7dd None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:16.037033 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090b7dd-5959-4cf6-9fad-1a286f94f7dd None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:16.074202 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4848568-dbf8-4c63-9a35-5123855f93e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:16.075200 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4848568-dbf8-4c63-9a35-5123855f93e4 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:16.075662 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4848568-dbf8-4c63-9a35-5123855f93e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:16.076113 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4848568-dbf8-4c63-9a35-5123855f93e4 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:16.077018 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4848568-dbf8-4c63-9a35-5123855f93e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:16.077521 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4848568-dbf8-4c63-9a35-5123855f93e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:16.079059 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4848568-dbf8-4c63-9a35-5123855f93e4 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:16.092140 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 414/838] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:16.097486 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f8577-aec4-4664-9ddc-1e9b9d077766 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:16.153691 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed586a7-5c16-49c0-9917-05ae62174c68 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:16.154042 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed586a7-5c16-49c0-9917-05ae62174c68 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:16.154280 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed586a7-5c16-49c0-9917-05ae62174c68 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:16.156425 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 415/839] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:33:16.161958 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337aa4dd-f92d-4f0e-879c-012cb421af92 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:16.162364 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337aa4dd-f92d-4f0e-879c-012cb421af92 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:16.163179 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337aa4dd-f92d-4f0e-879c-012cb421af92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:16.187028 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-337aa4dd-f92d-4f0e-879c-012cb421af92 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:16.188709 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-337aa4dd-f92d-4f0e-879c-012cb421af92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:16.214887 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f8577-aec4-4664-9ddc-1e9b9d077766 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:16.226338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:16.248417 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:16.255101 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac11ad-dcb1-44d8-a0c6-6af36914ef13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:16.255904 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac11ad-dcb1-44d8-a0c6-6af36914ef13 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:16.256189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac11ad-dcb1-44d8-a0c6-6af36914ef13 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:16.256466 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac11ad-dcb1-44d8-a0c6-6af36914ef13 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:16.269357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ac11ad-dcb1-44d8-a0c6-6af36914ef13 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:16.271838 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ac11ad-dcb1-44d8-a0c6-6af36914ef13 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:16.279148 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 416/840] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:16.288840 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-93ac11ad-dcb1-44d8-a0c6-6af36914ef13 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 06 13:33:16.288840 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:16.288840 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:16.288840 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:16.288840 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:16.288840 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:16.288840 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:16.288840 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:16.288840 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:16.288840 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:16.288840 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:16.288840 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:16.288840 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:33:16.288840 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:33:16.288840 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:33:16.288840 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:33:16.289801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:16.289801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:16.289801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:33:16.289801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:16.289801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:33:16.289801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:16.289801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:16.289801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:16.289801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:16.289801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:16.289801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:16.289801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:16.289801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:16.289801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:16.289801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:16.289801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:16.289801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:16.289801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:16.290754 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:16.290754 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:16.290754 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:33:16.290754 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:33:16.290754 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:33:16.290754 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:33:16.290754 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:33:16.290754 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:33:16.290754 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 06 13:33:16.290754 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:16.294125 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 425/841] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:33:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:16.304811 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337aa4dd-f92d-4f0e-879c-012cb421af92 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:16.313405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac11ad-dcb1-44d8-a0c6-6af36914ef13 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:16.419985 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337aa4dd-f92d-4f0e-879c-012cb421af92 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:16.433022 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac11ad-dcb1-44d8-a0c6-6af36914ef13 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:16.467578 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33440cb-87fe-49cd-9d09-26e06e03b6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:16.468181 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33440cb-87fe-49cd-9d09-26e06e03b6fb None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:16.468461 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33440cb-87fe-49cd-9d09-26e06e03b6fb None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:16.468735 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33440cb-87fe-49cd-9d09-26e06e03b6fb None admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:16.468985 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd8675-b01b-492a-83db-25bac48d0c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:16.469606 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33440cb-87fe-49cd-9d09-26e06e03b6fb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:16.469917 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33440cb-87fe-49cd-9d09-26e06e03b6fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:16.470452 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd8675-b01b-492a-83db-25bac48d0c61 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:16.470941 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd8675-b01b-492a-83db-25bac48d0c61 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:16.471339 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33440cb-87fe-49cd-9d09-26e06e03b6fb None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:16.471630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd8675-b01b-492a-83db-25bac48d0c61 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:16.472479 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd8675-b01b-492a-83db-25bac48d0c61 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:16.474066 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd8675-b01b-492a-83db-25bac48d0c61 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:16.486938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-f4dd8675-b01b-492a-83db-25bac48d0c61 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 06 13:33:16.486938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:16.486938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 13:33:16.486938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 13:33:16.486938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 13:33:16.486938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 13:33:16.486938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 13:33:16.486938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:16.486938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 06 13:33:16.486938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:16.486938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 13:33:16.486938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:16.486938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:16.486938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:16.486938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:16.486938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:16.486938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:16.487960 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:16.487960 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:16.487960 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:16.487960 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:16.487960 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:16.487960 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:16.487960 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 13:33:16.487960 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 13:33:16.487960 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 13:33:16.487960 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 13:33:16.487960 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:16.487960 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:16.487960 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:16.487960 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:16.487960 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:16.487960 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:16.488812 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:16.488812 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:16.488812 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:16.488812 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:16.488812 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:16.488812 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:16.488812 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:16.488812 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:16.488812 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:16.488812 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:16.488812 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 13:33:16.488812 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:16.488812 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 06 13:33:16.488812 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:16.495955 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 417/842] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Jul 6 13:33:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:16.496475 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 426/843] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jul 6 13:33:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:33:16.497475 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33440cb-87fe-49cd-9d09-26e06e03b6fb None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:16.498465 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd8675-b01b-492a-83db-25bac48d0c61 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:16.530315 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:16.562686 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:16.621530 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33440cb-87fe-49cd-9d09-26e06e03b6fb None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:16.627508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd8675-b01b-492a-83db-25bac48d0c61 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:16.665241 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3c6f1-837e-44e8-a04a-aeb7b09fad98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:16.667052 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3c6f1-837e-44e8-a04a-aeb7b09fad98 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:16.667463 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29869f16-ff1f-433a-8f22-216dd15eb653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:16.667957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3c6f1-837e-44e8-a04a-aeb7b09fad98 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:16.668135 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29869f16-ff1f-433a-8f22-216dd15eb653 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:16.668388 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29869f16-ff1f-433a-8f22-216dd15eb653 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:16.668661 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29869f16-ff1f-433a-8f22-216dd15eb653 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:16.669091 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3c6f1-837e-44e8-a04a-aeb7b09fad98 None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:16.670046 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a3c6f1-837e-44e8-a04a-aeb7b09fad98 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:16.670549 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a3c6f1-837e-44e8-a04a-aeb7b09fad98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:16.672111 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a3c6f1-837e-44e8-a04a-aeb7b09fad98 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:16.682092 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 427/844] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jul 6 13:33:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:16.683171 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3c6f1-837e-44e8-a04a-aeb7b09fad98 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:16.756905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29869f16-ff1f-433a-8f22-216dd15eb653 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:16.758392 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29869f16-ff1f-433a-8f22-216dd15eb653 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:16.789918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:16.791428 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3c6f1-837e-44e8-a04a-aeb7b09fad98 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:16.811868 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:16.831166 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f387f8-2bad-43e8-9f3f-4e7f60c2ce7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:16.832221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f387f8-2bad-43e8-9f3f-4e7f60c2ce7c None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:16.832708 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f387f8-2bad-43e8-9f3f-4e7f60c2ce7c None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:16.833226 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f387f8-2bad-43e8-9f3f-4e7f60c2ce7c None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:16.834152 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f387f8-2bad-43e8-9f3f-4e7f60c2ce7c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:16.834666 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f387f8-2bad-43e8-9f3f-4e7f60c2ce7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:16.836216 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f387f8-2bad-43e8-9f3f-4e7f60c2ce7c None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:16.849575 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 428/845] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jul 6 13:33:16 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:16.892332 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 418/846] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:33:16 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 13:33:16.894050 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f387f8-2bad-43e8-9f3f-4e7f60c2ce7c None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:17.018385 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f387f8-2bad-43e8-9f3f-4e7f60c2ce7c None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:17.054965 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066c889-c54b-4c82-8074-903a80147b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:17.056115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066c889-c54b-4c82-8074-903a80147b12 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:17.056634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066c889-c54b-4c82-8074-903a80147b12 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:17.057189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066c889-c54b-4c82-8074-903a80147b12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:17.068406 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066c889-c54b-4c82-8074-903a80147b12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:17.070152 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066c889-c54b-4c82-8074-903a80147b12 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:17.088764 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-e066c889-c54b-4c82-8074-903a80147b12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:17.088764 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:17.088764 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:17.088764 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:17.088764 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:17.088764 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:17.088764 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:17.088764 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:17.088764 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:17.088764 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:17.088764 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:17.088764 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:17.088764 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:17.088764 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:17.088764 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:17.089704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:17.089704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:17.089704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:17.089704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:17.089704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:17.089704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:17.089704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:17.089704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:17.089704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:17.089704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:17.089704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:17.089704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:17.089704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:17.089704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:17.089704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:17.089704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:17.089704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:17.090606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:17.090606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:17.090606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:17.090606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:17.092775 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 429/847] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:17.098077 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29869f16-ff1f-433a-8f22-216dd15eb653 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:17.203971 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29869f16-ff1f-433a-8f22-216dd15eb653 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:17.228104 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:17.239722 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ee401-3790-4d5f-9380-1c472b598d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:17.240389 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ee401-3790-4d5f-9380-1c472b598d28 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:17.240669 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ee401-3790-4d5f-9380-1c472b598d28 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:17.241068 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ee401-3790-4d5f-9380-1c472b598d28 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:17.242003 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ee401-3790-4d5f-9380-1c472b598d28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:17.242314 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ee401-3790-4d5f-9380-1c472b598d28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:17.243675 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ee401-3790-4d5f-9380-1c472b598d28 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:17.260542 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 419/848] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:17.261956 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066c889-c54b-4c82-8074-903a80147b12 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:17.303853 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:17.375124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066c889-c54b-4c82-8074-903a80147b12 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:17.410793 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed221efd-48cf-4525-b04c-683520cdb686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:17.411652 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed221efd-48cf-4525-b04c-683520cdb686 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:17.411936 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed221efd-48cf-4525-b04c-683520cdb686 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:17.412267 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed221efd-48cf-4525-b04c-683520cdb686 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:17.419971 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed221efd-48cf-4525-b04c-683520cdb686 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:17.421435 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed221efd-48cf-4525-b04c-683520cdb686 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:17.436177 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-ed221efd-48cf-4525-b04c-683520cdb686 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:17.436177 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:17.436177 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:17.436177 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:17.436177 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:17.436177 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:17.436177 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:17.436177 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:17.436177 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:17.436177 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:17.436177 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:17.436177 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:17.436177 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:33:17.436177 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:33:17.436177 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:33:17.437133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:33:17.437133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:17.437133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:17.437133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:17.437133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:17.437133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:17.437133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:17.437133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:17.437133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:17.437133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:17.437133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:17.437133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:17.437133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:17.437133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:17.437133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:17.437133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:17.437133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:17.438010 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:33:17.438010 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:33:17.438010 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:33:17.438010 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:33:17.438010 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:33:17.438010 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:33:17.438010 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:17.438010 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:17.441117 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 430/849] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:33:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:17.443457 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ee401-3790-4d5f-9380-1c472b598d28 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:17.553920 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ee401-3790-4d5f-9380-1c472b598d28 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:17.589019 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b610701-f9a3-4645-a051-dce5bbfcac81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:17.589731 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b610701-f9a3-4645-a051-dce5bbfcac81 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:17.590013 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b610701-f9a3-4645-a051-dce5bbfcac81 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:17.590289 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b610701-f9a3-4645-a051-dce5bbfcac81 None admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:17.591076 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b610701-f9a3-4645-a051-dce5bbfcac81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:17.591384 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b610701-f9a3-4645-a051-dce5bbfcac81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:17.592810 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b610701-f9a3-4645-a051-dce5bbfcac81 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:17.610835 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 420/850] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:33:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:17.612175 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed221efd-48cf-4525-b04c-683520cdb686 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:17.714139 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:17.720887 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed221efd-48cf-4525-b04c-683520cdb686 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:17.757942 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4770e7fe-00ba-4505-908b-f8cf12aaeba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:17.758602 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770e7fe-00ba-4505-908b-f8cf12aaeba3 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:17.758888 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770e7fe-00ba-4505-908b-f8cf12aaeba3 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:17.759168 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770e7fe-00ba-4505-908b-f8cf12aaeba3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:17.761135 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4770e7fe-00ba-4505-908b-f8cf12aaeba3 None admin] RBAC: Adding query filter params (role.id=795daa2c78354fd48e4a741e1182cc98, scope.project.id=bfe6c3d8da2743be9609184db2506303, user.id=8c1bea05c37646b0a05fa8dda0367ef2) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:17.761456 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4770e7fe-00ba-4505-908b-f8cf12aaeba3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:17.762888 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4770e7fe-00ba-4505-908b-f8cf12aaeba3 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:17.769319 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 431/851] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:33:17 2022] GET /identity/v3/role_assignments?user.id=8c1bea05c37646b0a05fa8dda0367ef2&scope.project.id=bfe6c3d8da2743be9609184db2506303&role.id=795daa2c78354fd48e4a741e1182cc98 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:17.891913 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404fe8b-c894-46cb-b858-7274660911d6 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:17.892438 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404fe8b-c894-46cb-b858-7274660911d6 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:17.892623 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404fe8b-c894-46cb-b858-7274660911d6 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:17.894565 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 421/852] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:17.900305 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e344a3-7986-457d-8698-d8a0cf80d2f5 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:17.901382 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e344a3-7986-457d-8698-d8a0cf80d2f5 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:17.901783 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e344a3-7986-457d-8698-d8a0cf80d2f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:17.920245 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-62e344a3-7986-457d-8698-d8a0cf80d2f5 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:17.921469 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-62e344a3-7986-457d-8698-d8a0cf80d2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:17.982773 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a5d8f-d2f8-4a37-99c7-cd8c317943b1 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:17.983402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a5d8f-d2f8-4a37-99c7-cd8c317943b1 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:17.983875 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a5d8f-d2f8-4a37-99c7-cd8c317943b1 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:17.985730 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 422/853] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:33:17.990772 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bfe346-4eb8-44c7-9173-33dfae14b042 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:17.991064 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bfe346-4eb8-44c7-9173-33dfae14b042 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:17.991341 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bfe346-4eb8-44c7-9173-33dfae14b042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:17.996633 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 432/854] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:18.012398 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-23bfe346-4eb8-44c7-9173-33dfae14b042 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:18.013617 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-23bfe346-4eb8-44c7-9173-33dfae14b042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:18.020743 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e344a3-7986-457d-8698-d8a0cf80d2f5 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:18.061408 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:18.091152 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 423/855] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:18.114702 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bfe346-4eb8-44c7-9173-33dfae14b042 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:18.139075 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e344a3-7986-457d-8698-d8a0cf80d2f5 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:18.153484 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:18.184826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a12e5e-cee9-4530-bf8a-22f73bc7febf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:18.185828 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a12e5e-cee9-4530-bf8a-22f73bc7febf None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:18.186119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a12e5e-cee9-4530-bf8a-22f73bc7febf None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:18.186397 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a12e5e-cee9-4530-bf8a-22f73bc7febf None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:18.187541 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a12e5e-cee9-4530-bf8a-22f73bc7febf None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:18.189187 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a12e5e-cee9-4530-bf8a-22f73bc7febf None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:18.204804 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-f4a12e5e-cee9-4530-bf8a-22f73bc7febf None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 06 13:33:18.204804 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:18.204804 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 13:33:18.204804 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 13:33:18.204804 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 13:33:18.204804 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 13:33:18.204804 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 13:33:18.204804 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:18.204804 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 06 13:33:18.204804 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:18.204804 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 13:33:18.204804 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:18.204804 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:18.204804 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:18.204804 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:18.204804 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:18.205782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:18.205782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:18.205782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:18.205782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:18.205782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:18.205782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:18.205782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:18.205782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 13:33:18.205782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 13:33:18.205782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 13:33:18.205782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 13:33:18.205782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:18.205782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:18.205782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:18.205782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:18.205782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:18.206623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:18.206623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:18.206623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:18.206623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:18.206623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:18.206623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:18.206623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:18.206623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:18.206623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:18.206623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:18.206623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:18.206623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 13:33:18.206623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:18.206623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 06 13:33:18.206623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:18.207430 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 433/856] 10.222.0.20 () {56 vars in 1132 bytes} [Wed Jul 6 13:33:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:33:18.214281 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a12e5e-cee9-4530-bf8a-22f73bc7febf None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:18.222521 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bfe346-4eb8-44c7-9173-33dfae14b042 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:18.262132 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29dcef-82df-41b5-9589-05cf2b8e01d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:18.262979 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29dcef-82df-41b5-9589-05cf2b8e01d9 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:18.263265 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29dcef-82df-41b5-9589-05cf2b8e01d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:18.263539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29dcef-82df-41b5-9589-05cf2b8e01d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:18.274160 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af29dcef-82df-41b5-9589-05cf2b8e01d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:33:18.282671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-af29dcef-82df-41b5-9589-05cf2b8e01d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:33:18.282671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:18.282671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:18.282671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:18.282671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:18.282671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:18.282671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:18.282671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:18.282671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:18.282671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:18.282671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:18.282671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:18.282671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:33:18.282671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:33:18.282671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:33:18.282671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:33:18.284929 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:33:18.284929 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:33:18.284929 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:33:18.284929 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:33:18.284929 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:18.284929 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:18.284929 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:18.284929 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:18.284929 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:18.284929 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:18.284929 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:18.284929 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:18.284929 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:18.284929 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:18.284929 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:18.284929 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:18.287030 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:18.287030 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:18.287030 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:18.287030 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:18.287030 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:33:18.287030 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:33:18.287030 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:33:18.287030 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:33:18.287030 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:33:18.287030 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:33:18.287030 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:33:18.287030 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:18.294009 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 424/857] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jul 6 13:33:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:33:18.296367 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29dcef-82df-41b5-9589-05cf2b8e01d9 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:18.337264 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:18.348755 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a12e5e-cee9-4530-bf8a-22f73bc7febf None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:18.394836 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605df914-bc2e-41c2-8098-03d602b6c12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:18.395512 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605df914-bc2e-41c2-8098-03d602b6c12b None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:18.395793 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605df914-bc2e-41c2-8098-03d602b6c12b None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:18.396071 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605df914-bc2e-41c2-8098-03d602b6c12b None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:18.396780 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605df914-bc2e-41c2-8098-03d602b6c12b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:18.397114 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605df914-bc2e-41c2-8098-03d602b6c12b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:18.398479 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605df914-bc2e-41c2-8098-03d602b6c12b None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:18.406656 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 434/858] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Jul 6 13:33:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:18.411286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605df914-bc2e-41c2-8098-03d602b6c12b None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:18.421500 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29dcef-82df-41b5-9589-05cf2b8e01d9 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:18.434702 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:18.472349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c12e2-0232-4bef-9d7d-007e35995f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:18.473940 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c12e2-0232-4bef-9d7d-007e35995f1b None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:18.474597 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c12e2-0232-4bef-9d7d-007e35995f1b None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:18.475138 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c12e2-0232-4bef-9d7d-007e35995f1b None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:18.476284 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2c12e2-0232-4bef-9d7d-007e35995f1b None admin] RBAC: Adding query filter params (name=service) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:18.476741 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2c12e2-0232-4bef-9d7d-007e35995f1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:18.479807 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2c12e2-0232-4bef-9d7d-007e35995f1b None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:18.496532 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 425/859] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jul 6 13:33:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:18.497977 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c12e2-0232-4bef-9d7d-007e35995f1b None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:18.530473 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605df914-bc2e-41c2-8098-03d602b6c12b None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:18.568511 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a95c69-4f85-4887-9a6a-228be994736d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:18.569278 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a95c69-4f85-4887-9a6a-228be994736d None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:18.569569 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a95c69-4f85-4887-9a6a-228be994736d None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:18.569848 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a95c69-4f85-4887-9a6a-228be994736d None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:18.570566 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a95c69-4f85-4887-9a6a-228be994736d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:18.570875 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a95c69-4f85-4887-9a6a-228be994736d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:18.572387 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a95c69-4f85-4887-9a6a-228be994736d None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:18.585182 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 435/860] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Jul 6 13:33:18 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:18.606665 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:18.615240 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c12e2-0232-4bef-9d7d-007e35995f1b None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:18.655202 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14658e-e973-4633-bfe7-616980445061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:18.656381 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14658e-e973-4633-bfe7-616980445061 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:18.656957 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14658e-e973-4633-bfe7-616980445061 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:18.657478 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14658e-e973-4633-bfe7-616980445061 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:18.671939 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e14658e-e973-4633-bfe7-616980445061 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:18.674045 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e14658e-e973-4633-bfe7-616980445061 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:18.692904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-3e14658e-e973-4633-bfe7-616980445061 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:18.692904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:18.692904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:18.692904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:18.692904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:18.692904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:18.692904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:18.692904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:18.692904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:18.692904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:18.692904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:18.692904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:18.692904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:18.692904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:18.692904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:18.694049 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:18.694049 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:18.694049 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:18.694049 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:18.694049 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:18.694049 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:18.694049 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:18.694049 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:18.694049 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:18.694049 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:18.694049 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:18.694049 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:18.694049 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:18.694049 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:18.694049 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:18.694049 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:18.694049 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:18.694950 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:18.694950 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:18.694950 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:18.694950 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:18.697255 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 426/861] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:18.703090 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a95c69-4f85-4887-9a6a-228be994736d None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:18.794797 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:18.815430 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a95c69-4f85-4887-9a6a-228be994736d None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:18.850945 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9cf59a-c6ad-439e-99d7-d6004f6b41d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:18.851617 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9cf59a-c6ad-439e-99d7-d6004f6b41d0 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:18.851947 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9cf59a-c6ad-439e-99d7-d6004f6b41d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:18.852234 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9cf59a-c6ad-439e-99d7-d6004f6b41d0 None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:18.852971 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9cf59a-c6ad-439e-99d7-d6004f6b41d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:18.853307 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9cf59a-c6ad-439e-99d7-d6004f6b41d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:18.855282 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9cf59a-c6ad-439e-99d7-d6004f6b41d0 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:18.867776 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 436/862] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:18.873585 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14658e-e973-4633-bfe7-616980445061 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:18.980430 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb36a96-c42a-4494-a701-d038b963bfb8 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:18.980623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb36a96-c42a-4494-a701-d038b963bfb8 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:18.980930 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb36a96-c42a-4494-a701-d038b963bfb8 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:18.982570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 437/863] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:18.982962 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14658e-e973-4633-bfe7-616980445061 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:18.989022 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba1546-4f9c-4148-bf28-b526cb755840 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:18.989022 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba1546-4f9c-4148-bf28-b526cb755840 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:18.989022 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba1546-4f9c-4148-bf28-b526cb755840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:19.006250 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-5bba1546-4f9c-4148-bf28-b526cb755840 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:19.007896 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-5bba1546-4f9c-4148-bf28-b526cb755840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:19.023664 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f446d-78cf-46cb-bad5-7ca620b8868a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:19.024777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f446d-78cf-46cb-bad5-7ca620b8868a None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:19.026090 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f446d-78cf-46cb-bad5-7ca620b8868a None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:19.026378 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f446d-78cf-46cb-bad5-7ca620b8868a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:19.036326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f446d-78cf-46cb-bad5-7ca620b8868a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:19.037958 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f446d-78cf-46cb-bad5-7ca620b8868a None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:19.058328 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-e27f446d-78cf-46cb-bad5-7ca620b8868a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 13:33:19.058328 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:19.058328 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:19.058328 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:19.058328 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:19.058328 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:19.058328 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:19.058328 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:19.058328 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:19.058328 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:19.058328 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:19.058328 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:19.058328 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:33:19.058328 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:33:19.058328 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:33:19.058328 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:33:19.059508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:19.059508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:19.059508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:33:19.059508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:19.059508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:33:19.059508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:19.059508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:19.059508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:19.059508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:19.059508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:19.059508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:19.059508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:19.059508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:19.059508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:19.059508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:19.059508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:19.059508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:19.059508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:19.060738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:19.060738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:19.060738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:33:19.060738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:33:19.060738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:33:19.060738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:33:19.060738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:33:19.060738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:33:19.060738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 13:33:19.060738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:19.063708 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 427/864] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Jul 6 13:33:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:19.064821 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f446d-78cf-46cb-bad5-7ca620b8868a None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:19.076080 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:19.099499 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 438/865] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:18 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:19.123476 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba1546-4f9c-4148-bf28-b526cb755840 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:19.175689 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f446d-78cf-46cb-bad5-7ca620b8868a None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:19.210981 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855fceda-2d39-4e45-9490-2563308f8ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:19.211963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fceda-2d39-4e45-9490-2563308f8ebd None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:19.212450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fceda-2d39-4e45-9490-2563308f8ebd None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:19.213046 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fceda-2d39-4e45-9490-2563308f8ebd None admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:19.214274 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855fceda-2d39-4e45-9490-2563308f8ebd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:19.214790 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855fceda-2d39-4e45-9490-2563308f8ebd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:19.216412 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855fceda-2d39-4e45-9490-2563308f8ebd None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:19.217334 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:19.241395 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 428/866] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Jul 6 13:33:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:19.242666 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855fceda-2d39-4e45-9490-2563308f8ebd None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:19.254390 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba1546-4f9c-4148-bf28-b526cb755840 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:19.308749 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0729fe38-e2dc-46bd-b0af-6a58f2f0b448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:19.312140 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729fe38-e2dc-46bd-b0af-6a58f2f0b448 None admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:19.312424 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729fe38-e2dc-46bd-b0af-6a58f2f0b448 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:19.312699 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729fe38-e2dc-46bd-b0af-6a58f2f0b448 None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:19.313470 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0729fe38-e2dc-46bd-b0af-6a58f2f0b448 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:19.314857 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0729fe38-e2dc-46bd-b0af-6a58f2f0b448 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:19.327763 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:19.337527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 439/867] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Jul 6 13:33:19 2022] POST /identity/v3/services => generated 251 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:33:19.371017 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855fceda-2d39-4e45-9490-2563308f8ebd None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:19.419149 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f73c1-30e1-42f2-b6f5-e464ae2a45e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:19.420196 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f73c1-30e1-42f2-b6f5-e464ae2a45e5 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:19.420682 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f73c1-30e1-42f2-b6f5-e464ae2a45e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:19.421207 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f73c1-30e1-42f2-b6f5-e464ae2a45e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:19.503498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f73c1-30e1-42f2-b6f5-e464ae2a45e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:19.504885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f73c1-30e1-42f2-b6f5-e464ae2a45e5 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:19.570543 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:19.615286 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 429/868] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:33:19 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 13:33:19.621924 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0729fe38-e2dc-46bd-b0af-6a58f2f0b448 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:19.733782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0729fe38-e2dc-46bd-b0af-6a58f2f0b448 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:19.767183 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:19.773866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d993d48f-2af5-4f01-bf39-bf4de957295f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:19.774681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993d48f-2af5-4f01-bf39-bf4de957295f None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:19.774956 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993d48f-2af5-4f01-bf39-bf4de957295f None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:19.775818 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993d48f-2af5-4f01-bf39-bf4de957295f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:19.787640 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d993d48f-2af5-4f01-bf39-bf4de957295f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:19.789127 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d993d48f-2af5-4f01-bf39-bf4de957295f None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:19.808952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-d993d48f-2af5-4f01-bf39-bf4de957295f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:19.808952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:19.808952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:19.808952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:19.808952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:19.808952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:19.808952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:19.808952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:19.808952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:19.808952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:19.808952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:19.808952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:19.808952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:19.808952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:19.808952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:19.810097 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:19.810097 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:19.810097 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:19.810097 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:19.810097 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:19.810097 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:19.810097 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:19.810097 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:19.810097 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:19.810097 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:19.810097 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:19.810097 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:19.810097 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:19.810097 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:19.810097 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:19.810097 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:19.810097 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:19.811208 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:19.811208 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:19.811208 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:19.811208 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:19.813170 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 440/869] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:19.818926 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f73c1-30e1-42f2-b6f5-e464ae2a45e5 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:19.917976 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33c084-a7f5-4c06-ab55-9e43a937406c None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:19.918915 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33c084-a7f5-4c06-ab55-9e43a937406c None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:19.919411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33c084-a7f5-4c06-ab55-9e43a937406c None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:19.922171 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 441/870] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:19.929047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f73c1-30e1-42f2-b6f5-e464ae2a45e5 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:19.930491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47252b4-c986-4c65-b902-403baefba498 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:19.931048 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47252b4-c986-4c65-b902-403baefba498 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:19.931530 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47252b4-c986-4c65-b902-403baefba498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:19.950156 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-b47252b4-c986-4c65-b902-403baefba498 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:19.951662 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-b47252b4-c986-4c65-b902-403baefba498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:19.965638 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec5b482-d972-4fe3-8c31-68edcd3e09f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:19.966354 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5b482-d972-4fe3-8c31-68edcd3e09f5 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:19.966643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5b482-d972-4fe3-8c31-68edcd3e09f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:19.966916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5b482-d972-4fe3-8c31-68edcd3e09f5 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:19.967926 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec5b482-d972-4fe3-8c31-68edcd3e09f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:19.968340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec5b482-d972-4fe3-8c31-68edcd3e09f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:19.969857 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec5b482-d972-4fe3-8c31-68edcd3e09f5 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:19.983544 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 430/871] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:19.989967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec5b482-d972-4fe3-8c31-68edcd3e09f5 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:20.020534 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:20.044339 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 442/872] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:19 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:20.067922 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47252b4-c986-4c65-b902-403baefba498 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:20.110542 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec5b482-d972-4fe3-8c31-68edcd3e09f5 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:20.147597 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bebd22-3482-4e01-b561-b9b2b3db8310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:20.148769 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bebd22-3482-4e01-b561-b9b2b3db8310 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:20.149322 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bebd22-3482-4e01-b561-b9b2b3db8310 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:20.149796 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bebd22-3482-4e01-b561-b9b2b3db8310 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:20.157908 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bebd22-3482-4e01-b561-b9b2b3db8310 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:20.159867 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bebd22-3482-4e01-b561-b9b2b3db8310 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:20.176138 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-55bebd22-3482-4e01-b561-b9b2b3db8310 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:20.176138 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:20.176138 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:20.176138 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:20.176138 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:20.176138 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:20.176138 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:20.176138 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:20.176138 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:20.176138 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:20.176138 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:20.176138 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:20.176138 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:33:20.176138 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:33:20.176138 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:33:20.177127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:33:20.177127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:20.177127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:20.177127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:20.177127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:20.177127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:20.177127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:20.177127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:20.177127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:20.177127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:20.177127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:20.177127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:20.177127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:20.177127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:20.177127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:20.177127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:20.177127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:20.178024 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:33:20.178024 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:33:20.178024 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:33:20.178024 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:33:20.178024 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:33:20.178024 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:33:20.178024 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:20.178024 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:20.178448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47252b4-c986-4c65-b902-403baefba498 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:20.180718 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 431/873] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:33:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:20.189741 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bebd22-3482-4e01-b561-b9b2b3db8310 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:20.197302 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:20.221480 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14656b15-2b71-40a8-924b-d697b4aa156e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:20.222728 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14656b15-2b71-40a8-924b-d697b4aa156e None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:20.223213 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14656b15-2b71-40a8-924b-d697b4aa156e None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:20.223687 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14656b15-2b71-40a8-924b-d697b4aa156e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:20.224542 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14656b15-2b71-40a8-924b-d697b4aa156e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:20.226129 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14656b15-2b71-40a8-924b-d697b4aa156e None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:20.239615 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-14656b15-2b71-40a8-924b-d697b4aa156e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 13:33:20.239615 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:20.239615 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 13:33:20.239615 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 13:33:20.239615 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 13:33:20.239615 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 13:33:20.239615 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 13:33:20.239615 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:20.239615 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 13:33:20.239615 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:20.239615 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 13:33:20.239615 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:20.239615 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:20.239615 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:20.239615 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:20.239615 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:20.239615 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:20.240700 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:20.240700 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:20.240700 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:20.240700 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:20.240700 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:20.240700 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:20.240700 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 13:33:20.240700 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 13:33:20.240700 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 13:33:20.240700 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 13:33:20.240700 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:20.240700 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:20.240700 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:20.240700 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:20.240700 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:20.240700 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:20.241953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:20.241953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:20.241953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:20.241953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:20.241953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:20.241953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:20.241953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:20.241953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:20.241953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:20.241953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:20.241953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 13:33:20.241953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:20.241953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 13:33:20.241953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:20.242686 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 443/874] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Jul 6 13:33:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:33:20.253852 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14656b15-2b71-40a8-924b-d697b4aa156e None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:20.285685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:20.310922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bebd22-3482-4e01-b561-b9b2b3db8310 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:20.353505 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c54745-51ee-41de-a1dc-1e344c79f2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:20.354202 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c54745-51ee-41de-a1dc-1e344c79f2d3 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:20.354496 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c54745-51ee-41de-a1dc-1e344c79f2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:20.354910 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c54745-51ee-41de-a1dc-1e344c79f2d3 None admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:20.355704 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c54745-51ee-41de-a1dc-1e344c79f2d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:20.356381 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c54745-51ee-41de-a1dc-1e344c79f2d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:20.358206 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c54745-51ee-41de-a1dc-1e344c79f2d3 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:20.378991 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 432/875] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:33:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:20.380253 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14656b15-2b71-40a8-924b-d697b4aa156e None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:20.381239 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c54745-51ee-41de-a1dc-1e344c79f2d3 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:20.429673 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c2aa93-8d75-42a3-9c1f-2aaa15db648a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:20.431185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2aa93-8d75-42a3-9c1f-2aaa15db648a None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:20.432458 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2aa93-8d75-42a3-9c1f-2aaa15db648a None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:20.433451 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2aa93-8d75-42a3-9c1f-2aaa15db648a None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:20.434765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c2aa93-8d75-42a3-9c1f-2aaa15db648a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:20.435275 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c2aa93-8d75-42a3-9c1f-2aaa15db648a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:20.437351 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c2aa93-8d75-42a3-9c1f-2aaa15db648a None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:20.444476 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 444/876] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Jul 6 13:33:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:20.450277 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c2aa93-8d75-42a3-9c1f-2aaa15db648a None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:20.494504 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:20.516279 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c54745-51ee-41de-a1dc-1e344c79f2d3 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:20.553471 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa64d8b-60ec-4d15-b75c-f39d2620e2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:20.554204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa64d8b-60ec-4d15-b75c-f39d2620e2a2 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:20.554316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa64d8b-60ec-4d15-b75c-f39d2620e2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:20.554586 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa64d8b-60ec-4d15-b75c-f39d2620e2a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:20.555415 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa64d8b-60ec-4d15-b75c-f39d2620e2a2 None admin] RBAC: Adding query filter params (role.id=a6853d485f7942259b3646de3f386852, scope.project.id=bfe6c3d8da2743be9609184db2506303, user.id=c4535134181e489fb532f617264d5855) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:20.555726 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa64d8b-60ec-4d15-b75c-f39d2620e2a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:20.557297 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa64d8b-60ec-4d15-b75c-f39d2620e2a2 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:20.573174 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 433/877] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:33:20 2022] GET /identity/v3/role_assignments?user.id=c4535134181e489fb532f617264d5855&scope.project.id=bfe6c3d8da2743be9609184db2506303&role.id=a6853d485f7942259b3646de3f386852 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:20.574746 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c2aa93-8d75-42a3-9c1f-2aaa15db648a None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:20.617245 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d0047-e325-494a-87f9-406a22427d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:20.618339 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d0047-e325-494a-87f9-406a22427d5a None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:20.619418 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d0047-e325-494a-87f9-406a22427d5a None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:20.620167 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d0047-e325-494a-87f9-406a22427d5a None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:20.621296 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d0047-e325-494a-87f9-406a22427d5a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:20.621829 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d0047-e325-494a-87f9-406a22427d5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:20.623470 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d0047-e325-494a-87f9-406a22427d5a None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:20.630200 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 445/878] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Jul 6 13:33:20 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:20.696807 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1f5b9-02ca-44a4-ae11-3cc86b5cfe5e None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:20.697659 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1f5b9-02ca-44a4-ae11-3cc86b5cfe5e None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:20.697968 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1f5b9-02ca-44a4-ae11-3cc86b5cfe5e None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:20.699704 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 434/879] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:20.705421 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02200bb5-73ca-41e4-b433-7cbe58fc08ef None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:20.706044 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02200bb5-73ca-41e4-b433-7cbe58fc08ef None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:20.706518 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02200bb5-73ca-41e4-b433-7cbe58fc08ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:20.726685 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-02200bb5-73ca-41e4-b433-7cbe58fc08ef None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:20.728780 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-02200bb5-73ca-41e4-b433-7cbe58fc08ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:20.807080 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 446/880] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:20 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 13:33:20.829675 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1f5b9-02ca-44a4-ae11-3cc86b5cfe5e None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:20.840121 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:20.941131 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1f5b9-02ca-44a4-ae11-3cc86b5cfe5e None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:20.977359 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed9b0cf-9d42-43d6-8b1b-a1f8e7e6d557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:20.979043 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9b0cf-9d42-43d6-8b1b-a1f8e7e6d557 None admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:20.979569 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9b0cf-9d42-43d6-8b1b-a1f8e7e6d557 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:20.979992 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9b0cf-9d42-43d6-8b1b-a1f8e7e6d557 None admin] PATH_INFO: `/v3/services` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:20.980868 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed9b0cf-9d42-43d6-8b1b-a1f8e7e6d557 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:20.982498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed9b0cf-9d42-43d6-8b1b-a1f8e7e6d557 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:20.999700 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 435/881] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Jul 6 13:33:20 2022] POST /identity/v3/services => generated 269 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:33:21.552297 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b004b-86cc-4c1e-b9fa-538cac6ea317 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:21.552640 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b004b-86cc-4c1e-b9fa-538cac6ea317 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:21.552811 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b004b-86cc-4c1e-b9fa-538cac6ea317 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:21.554401 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 447/882] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:21.560149 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33619c37-a94d-4d06-9226-e1f78684d26f None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:21.560448 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33619c37-a94d-4d06-9226-e1f78684d26f None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:21.560731 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33619c37-a94d-4d06-9226-e1f78684d26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:21.579128 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-33619c37-a94d-4d06-9226-e1f78684d26f None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:21.584434 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-33619c37-a94d-4d06-9226-e1f78684d26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:21.650860 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:21.664096 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 436/883] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:21 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:21.687009 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20b004b-86cc-4c1e-b9fa-538cac6ea317 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:21.723781 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:21.909373 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20b004b-86cc-4c1e-b9fa-538cac6ea317 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:21.944413 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3855c-70f6-45fe-b854-dca61653f631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:21.945267 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3855c-70f6-45fe-b854-dca61653f631 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:21.945552 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3855c-70f6-45fe-b854-dca61653f631 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:21.945825 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3855c-70f6-45fe-b854-dca61653f631 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:21.947171 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3855c-70f6-45fe-b854-dca61653f631 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:21.948285 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3855c-70f6-45fe-b854-dca61653f631 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:21.960189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-3dc3855c-70f6-45fe-b854-dca61653f631 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 06 13:33:21.960189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:21.960189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 13:33:21.960189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 13:33:21.960189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 13:33:21.960189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 13:33:21.960189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 13:33:21.960189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:21.960189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 06 13:33:21.960189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:21.960189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 13:33:21.960189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:21.960189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:21.960189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:21.960189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:21.960189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:21.960189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:21.961212 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:21.961212 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:21.961212 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:21.961212 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:21.961212 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:21.961212 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:21.961212 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 13:33:21.961212 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 13:33:21.961212 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 13:33:21.961212 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 13:33:21.961212 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:21.961212 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:21.961212 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:21.961212 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:21.961212 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:21.961212 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:21.962073 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:21.962073 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:21.962073 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:21.962073 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:21.962073 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:21.962073 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:21.962073 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:21.962073 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:21.962073 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:21.962073 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:21.962073 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 13:33:21.962073 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:21.962073 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 06 13:33:21.962073 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:21.962073 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 448/884] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jul 6 13:33:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:21.968100 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33619c37-a94d-4d06-9226-e1f78684d26f None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:22.068588 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33619c37-a94d-4d06-9226-e1f78684d26f None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:22.103001 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e88c299-1ed6-4255-9d48-81f91bf0240b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:22.103721 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88c299-1ed6-4255-9d48-81f91bf0240b None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:22.104002 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88c299-1ed6-4255-9d48-81f91bf0240b None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:22.104279 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88c299-1ed6-4255-9d48-81f91bf0240b None admin] PATH_INFO: `/v3/services` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:22.104987 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e88c299-1ed6-4255-9d48-81f91bf0240b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:22.105332 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e88c299-1ed6-4255-9d48-81f91bf0240b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:22.106696 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e88c299-1ed6-4255-9d48-81f91bf0240b None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:22.111896 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 437/885] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jul 6 13:33:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:22.117469 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3855c-70f6-45fe-b854-dca61653f631 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:22.219736 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:22.223510 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3855c-70f6-45fe-b854-dca61653f631 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:22.259397 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c9ba08-b413-4916-9282-4a52a6f81a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:22.260060 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c9ba08-b413-4916-9282-4a52a6f81a4a None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:22.260339 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c9ba08-b413-4916-9282-4a52a6f81a4a None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:22.260616 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c9ba08-b413-4916-9282-4a52a6f81a4a None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:22.261353 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c9ba08-b413-4916-9282-4a52a6f81a4a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:22.261657 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c9ba08-b413-4916-9282-4a52a6f81a4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:22.263003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c9ba08-b413-4916-9282-4a52a6f81a4a None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:22.268216 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 449/886] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jul 6 13:33:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:22.273663 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e88c299-1ed6-4255-9d48-81f91bf0240b None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:22.333822 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:22.380399 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e88c299-1ed6-4255-9d48-81f91bf0240b None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:22.419508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85996b32-9141-42d0-81ac-951c7fd4ddf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:22.420495 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85996b32-9141-42d0-81ac-951c7fd4ddf6 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:22.420955 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85996b32-9141-42d0-81ac-951c7fd4ddf6 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:22.421440 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85996b32-9141-42d0-81ac-951c7fd4ddf6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:22.422399 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85996b32-9141-42d0-81ac-951c7fd4ddf6 None admin] RBAC: Adding query filter params (interface=public, service_id=a89bb7850b14458493648d357c7d4ebf, region_id=RegionOne) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:22.422884 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85996b32-9141-42d0-81ac-951c7fd4ddf6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:22.424680 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85996b32-9141-42d0-81ac-951c7fd4ddf6 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:22.430557 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 438/887] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Jul 6 13:33:22 2022] GET /identity/v3/endpoints?service_id=a89bb7850b14458493648d357c7d4ebf&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:22.439030 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c9ba08-b413-4916-9282-4a52a6f81a4a None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:22.535873 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c9ba08-b413-4916-9282-4a52a6f81a4a None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:22.570506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3409478-a0f5-453b-8bb7-6c238f22fb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:22.571227 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3409478-a0f5-453b-8bb7-6c238f22fb16 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:22.571512 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3409478-a0f5-453b-8bb7-6c238f22fb16 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:22.571788 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3409478-a0f5-453b-8bb7-6c238f22fb16 None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:22.572692 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3409478-a0f5-453b-8bb7-6c238f22fb16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:22.574081 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3409478-a0f5-453b-8bb7-6c238f22fb16 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:22.582795 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 450/888] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Jul 6 13:33:22 2022] GET /identity/v3/services => generated 1268 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:33:22.788777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f252f5-e22e-4f3b-beed-eb9e14b60c49 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:22.789068 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f252f5-e22e-4f3b-beed-eb9e14b60c49 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:22.789295 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f252f5-e22e-4f3b-beed-eb9e14b60c49 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:22.791151 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 439/889] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:22.796664 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6e1d0-0e76-4e4b-a974-c477b1f69503 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:22.797191 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6e1d0-0e76-4e4b-a974-c477b1f69503 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:22.797279 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6e1d0-0e76-4e4b-a974-c477b1f69503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:22.814454 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-71b6e1d0-0e76-4e4b-a974-c477b1f69503 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:22.815815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-71b6e1d0-0e76-4e4b-a974-c477b1f69503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:22.837341 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:22.886019 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 451/890] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:22.908650 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f252f5-e22e-4f3b-beed-eb9e14b60c49 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:22.930565 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7e234-f21d-438c-9562-a6a14a5d28d7 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:22.930902 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7e234-f21d-438c-9562-a6a14a5d28d7 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:22.931130 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7e234-f21d-438c-9562-a6a14a5d28d7 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:22.932950 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 452/891] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:33:22.943550 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4356230-7b80-48f8-b08e-2144f5ea8cc2 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:22.944354 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4356230-7b80-48f8-b08e-2144f5ea8cc2 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:22.944749 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4356230-7b80-48f8-b08e-2144f5ea8cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:22.967338 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-c4356230-7b80-48f8-b08e-2144f5ea8cc2 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:22.968853 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-c4356230-7b80-48f8-b08e-2144f5ea8cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:23.034376 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f252f5-e22e-4f3b-beed-eb9e14b60c49 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:23.049282 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 453/892] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 13:33:23.056373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:23.068115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4356230-7b80-48f8-b08e-2144f5ea8cc2 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:23.073860 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7645b91-a49c-4706-ac77-823bbab31f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:23.075147 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7645b91-a49c-4706-ac77-823bbab31f06 None admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:23.075437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7645b91-a49c-4706-ac77-823bbab31f06 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:23.075714 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7645b91-a49c-4706-ac77-823bbab31f06 None admin] PATH_INFO: `/v3/services` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:23.076455 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7645b91-a49c-4706-ac77-823bbab31f06 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:23.077917 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7645b91-a49c-4706-ac77-823bbab31f06 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:23.096853 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 440/893] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Jul 6 13:33:22 2022] POST /identity/v3/services => generated 241 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:33:23.116311 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991e161-564b-40cf-88c9-7f44f1e82dbb None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:23.116653 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991e161-564b-40cf-88c9-7f44f1e82dbb None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:23.116937 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991e161-564b-40cf-88c9-7f44f1e82dbb None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:23.118847 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 441/894] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:33:23.125177 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191882a-fb15-4987-a220-62d28367486b None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:23.125797 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191882a-fb15-4987-a220-62d28367486b None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:23.126278 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191882a-fb15-4987-a220-62d28367486b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:23.157015 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-7191882a-fb15-4987-a220-62d28367486b None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:23.158261 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-7191882a-fb15-4987-a220-62d28367486b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:23.169066 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:23.188362 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4356230-7b80-48f8-b08e-2144f5ea8cc2 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:23.244930 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be2b262-26c4-420c-a754-5ce8996c7d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:23.246177 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2b262-26c4-420c-a754-5ce8996c7d25 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:23.246461 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2b262-26c4-420c-a754-5ce8996c7d25 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:23.247382 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2b262-26c4-420c-a754-5ce8996c7d25 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:23.256704 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 442/895] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:23.259246 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be2b262-26c4-420c-a754-5ce8996c7d25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:33:23.267856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-8be2b262-26c4-420c-a754-5ce8996c7d25 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 13:33:23.267856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:23.267856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:23.267856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:23.267856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:23.267856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:23.267856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:23.267856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:23.267856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:23.267856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:23.267856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:23.267856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:23.267856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:33:23.267856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:33:23.267856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:33:23.267856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:33:23.268867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:33:23.268867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:33:23.268867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:33:23.268867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:33:23.268867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:23.268867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:23.268867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:23.268867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:23.268867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:23.268867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:23.268867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:23.268867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:23.268867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:23.268867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:23.268867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:23.268867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:23.269823 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:23.269823 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:23.269823 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:23.269823 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:23.269823 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:33:23.269823 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:33:23.269823 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:33:23.269823 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:33:23.269823 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:33:23.269823 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:33:23.269823 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 13:33:23.269823 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:23.273331 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 454/896] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Jul 6 13:33:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:23.275843 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191882a-fb15-4987-a220-62d28367486b None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:23.282804 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be2b262-26c4-420c-a754-5ce8996c7d25 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:23.395385 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191882a-fb15-4987-a220-62d28367486b None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:23.407012 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:23.413770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be2b262-26c4-420c-a754-5ce8996c7d25 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:23.435750 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a746723f-2ceb-4188-969a-02c2b4273f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:23.436793 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746723f-2ceb-4188-969a-02c2b4273f03 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:23.437011 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746723f-2ceb-4188-969a-02c2b4273f03 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:23.437362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746723f-2ceb-4188-969a-02c2b4273f03 None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:23.438211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a746723f-2ceb-4188-969a-02c2b4273f03 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:23.438584 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a746723f-2ceb-4188-969a-02c2b4273f03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:23.440019 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a746723f-2ceb-4188-969a-02c2b4273f03 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:23.454631 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 443/897] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Jul 6 13:33:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:23.455817 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a746723f-2ceb-4188-969a-02c2b4273f03 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:23.456556 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c9aa5-36bf-4feb-8d57-a460e4d933dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:23.457569 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c9aa5-36bf-4feb-8d57-a460e4d933dd None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:23.458009 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c9aa5-36bf-4feb-8d57-a460e4d933dd None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:23.458428 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c9aa5-36bf-4feb-8d57-a460e4d933dd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:23.459402 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c9aa5-36bf-4feb-8d57-a460e4d933dd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:23.460989 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c9aa5-36bf-4feb-8d57-a460e4d933dd None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:23.475384 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-311c9aa5-36bf-4feb-8d57-a460e4d933dd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 06 13:33:23.475384 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:23.475384 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 13:33:23.475384 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 13:33:23.475384 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 13:33:23.475384 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 13:33:23.475384 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 13:33:23.475384 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:23.475384 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 06 13:33:23.475384 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:23.475384 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 13:33:23.475384 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:23.475384 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:23.475384 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:23.475384 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:23.475384 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:23.476565 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:23.476565 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:23.476565 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:23.476565 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:23.476565 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:23.476565 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:23.476565 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:23.476565 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 13:33:23.476565 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 13:33:23.476565 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 13:33:23.476565 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 13:33:23.476565 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:23.476565 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:23.476565 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:23.476565 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:23.476565 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:23.477547 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:23.477547 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:23.477547 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:23.477547 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:23.477547 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:23.477547 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:23.477547 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:23.477547 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:23.477547 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:23.477547 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:23.477547 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:23.477547 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 13:33:23.477547 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:23.477547 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 06 13:33:23.477547 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:23.481359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 455/898] 10.222.0.20 () {56 vars in 1132 bytes} [Wed Jul 6 13:33:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:23.488379 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c9aa5-36bf-4feb-8d57-a460e4d933dd None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:23.499463 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:23.594054 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a746723f-2ceb-4188-969a-02c2b4273f03 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:23.602907 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c9aa5-36bf-4feb-8d57-a460e4d933dd None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:23.639093 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c2198-e294-41ac-9bd2-b9dc99cde66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:23.639650 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f122a-e5a5-4326-824d-706e385e9cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:23.640229 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c2198-e294-41ac-9bd2-b9dc99cde66d None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:23.640633 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f122a-e5a5-4326-824d-706e385e9cf1 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:23.640807 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c2198-e294-41ac-9bd2-b9dc99cde66d None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:23.641122 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f122a-e5a5-4326-824d-706e385e9cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:23.641428 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c2198-e294-41ac-9bd2-b9dc99cde66d None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:23.641545 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f122a-e5a5-4326-824d-706e385e9cf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:23.642541 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c2198-e294-41ac-9bd2-b9dc99cde66d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:23.642996 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c2198-e294-41ac-9bd2-b9dc99cde66d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:23.644782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c2198-e294-41ac-9bd2-b9dc99cde66d None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:23.651192 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 456/899] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Jul 6 13:33:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:23.656683 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c2198-e294-41ac-9bd2-b9dc99cde66d None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:23.662911 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f122a-e5a5-4326-824d-706e385e9cf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:23.663946 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f122a-e5a5-4326-824d-706e385e9cf1 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:23.689315 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-e63f122a-e5a5-4326-824d-706e385e9cf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:23.689315 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:23.689315 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:23.689315 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:23.689315 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:23.689315 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:23.689315 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:23.689315 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:23.689315 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:23.689315 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:23.689315 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:23.689315 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:23.689315 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:23.689315 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:23.689315 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:23.690260 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:23.690260 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:23.690260 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:23.690260 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:23.690260 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:23.690260 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:23.690260 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:23.690260 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:23.690260 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:23.690260 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:23.690260 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:23.690260 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:23.690260 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:23.690260 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:23.690260 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:23.690260 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:23.690260 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:23.691146 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:23.691146 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:23.691146 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:23.691146 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:23.693987 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 444/900] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:33:23.702325 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f122a-e5a5-4326-824d-706e385e9cf1 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:23.726518 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:23.737696 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:23.771713 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c2198-e294-41ac-9bd2-b9dc99cde66d None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:23.807525 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e499390-6adb-4834-902a-546b1fc9c95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:23.808188 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e499390-6adb-4834-902a-546b1fc9c95c None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:23.808536 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e499390-6adb-4834-902a-546b1fc9c95c None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:23.808827 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e499390-6adb-4834-902a-546b1fc9c95c None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:23.809635 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e499390-6adb-4834-902a-546b1fc9c95c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:23.809868 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e499390-6adb-4834-902a-546b1fc9c95c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:23.811300 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e499390-6adb-4834-902a-546b1fc9c95c None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:23.817032 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 457/901] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Jul 6 13:33:23 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:23.823473 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e499390-6adb-4834-902a-546b1fc9c95c None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:23.823751 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f122a-e5a5-4326-824d-706e385e9cf1 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:23.866645 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f2291-5d88-484a-8a9c-1a3161b2df29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:23.867770 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f2291-5d88-484a-8a9c-1a3161b2df29 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:23.868251 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f2291-5d88-484a-8a9c-1a3161b2df29 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:23.869602 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f2291-5d88-484a-8a9c-1a3161b2df29 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:23.870548 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2f2291-5d88-484a-8a9c-1a3161b2df29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:23.871050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2f2291-5d88-484a-8a9c-1a3161b2df29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:23.872614 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2f2291-5d88-484a-8a9c-1a3161b2df29 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:23.889409 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 445/902] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:23.894561 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f2291-5d88-484a-8a9c-1a3161b2df29 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:23.952536 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e499390-6adb-4834-902a-546b1fc9c95c None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:23.991411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fcef5b-ef30-432c-9a58-05c0c7cacba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:23.991620 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcef5b-ef30-432c-9a58-05c0c7cacba5 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:23.991728 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcef5b-ef30-432c-9a58-05c0c7cacba5 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:23.992005 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcef5b-ef30-432c-9a58-05c0c7cacba5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:23.992858 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fcef5b-ef30-432c-9a58-05c0c7cacba5 None admin] RBAC: Adding query filter params (interface=public, service_id=e15203638ff84097ab6c54b2aa9feca8, region_id=RegionOne) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:23.993208 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fcef5b-ef30-432c-9a58-05c0c7cacba5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:23.994905 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fcef5b-ef30-432c-9a58-05c0c7cacba5 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:23.996818 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:24.004945 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 458/903] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Jul 6 13:33:23 2022] GET /identity/v3/endpoints?service_id=e15203638ff84097ab6c54b2aa9feca8&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:24.014532 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fcef5b-ef30-432c-9a58-05c0c7cacba5 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:24.014703 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:24.032714 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f2291-5d88-484a-8a9c-1a3161b2df29 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:24.073724 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc68f5-33da-4e4f-87f0-75594d5361fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:24.074659 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc68f5-33da-4e4f-87f0-75594d5361fc None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:24.075211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc68f5-33da-4e4f-87f0-75594d5361fc None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:24.075295 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc68f5-33da-4e4f-87f0-75594d5361fc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:24.082944 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cc68f5-33da-4e4f-87f0-75594d5361fc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:24.084692 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cc68f5-33da-4e4f-87f0-75594d5361fc None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:24.101849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-d4cc68f5-33da-4e4f-87f0-75594d5361fc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 13:33:24.101849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:24.101849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:24.101849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:24.101849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:24.101849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:24.101849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:24.101849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:24.101849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:24.101849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:24.101849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:24.101849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:24.101849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:33:24.101849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:33:24.101849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:33:24.101849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:33:24.102845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:24.102845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:24.102845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:33:24.102845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:24.102845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:33:24.102845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:24.102845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:24.102845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:24.102845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:24.102845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:24.102845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:24.102845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:24.102845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:24.102845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:24.102845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:24.102845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:24.102845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:24.102845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:24.104322 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:24.104322 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:24.104322 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:33:24.104322 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:33:24.104322 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:33:24.104322 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:33:24.104322 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:33:24.104322 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:33:24.104322 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 13:33:24.104322 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:24.108302 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 446/904] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Jul 6 13:33:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:33:24.109665 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc68f5-33da-4e4f-87f0-75594d5361fc None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:24.149322 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fcef5b-ef30-432c-9a58-05c0c7cacba5 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:24.201688 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52139dea-1a3d-42e8-9a3e-bcded6f15132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:24.203362 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52139dea-1a3d-42e8-9a3e-bcded6f15132 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:24.203832 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52139dea-1a3d-42e8-9a3e-bcded6f15132 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:24.204302 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52139dea-1a3d-42e8-9a3e-bcded6f15132 None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:24.205206 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52139dea-1a3d-42e8-9a3e-bcded6f15132 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:24.207278 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52139dea-1a3d-42e8-9a3e-bcded6f15132 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:24.221055 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc68f5-33da-4e4f-87f0-75594d5361fc None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:24.221518 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 459/905] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Jul 6 13:33:24 2022] GET /identity/v3/services => generated 1497 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:33:24.265433 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604e85e-8ae4-4396-b2b8-42c36b8a0740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:24.266390 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604e85e-8ae4-4396-b2b8-42c36b8a0740 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:24.266849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604e85e-8ae4-4396-b2b8-42c36b8a0740 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:24.267295 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604e85e-8ae4-4396-b2b8-42c36b8a0740 None admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:24.268316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9604e85e-8ae4-4396-b2b8-42c36b8a0740 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:24.268791 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9604e85e-8ae4-4396-b2b8-42c36b8a0740 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:24.270426 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9604e85e-8ae4-4396-b2b8-42c36b8a0740 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:24.295392 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 447/906] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Jul 6 13:33:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:24.300507 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52139dea-1a3d-42e8-9a3e-bcded6f15132 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:24.379728 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:24.431434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52139dea-1a3d-42e8-9a3e-bcded6f15132 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:24.469593 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c575b3bd-95de-4daa-9ff9-5b6dd38ebc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:24.470485 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c575b3bd-95de-4daa-9ff9-5b6dd38ebc64 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:24.470931 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c575b3bd-95de-4daa-9ff9-5b6dd38ebc64 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:24.471364 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c575b3bd-95de-4daa-9ff9-5b6dd38ebc64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:24.542373 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c575b3bd-95de-4daa-9ff9-5b6dd38ebc64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:24.543823 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c575b3bd-95de-4daa-9ff9-5b6dd38ebc64 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:24.605545 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:24.652960 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 460/907] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:33:24 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 13:33:24.658307 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604e85e-8ae4-4396-b2b8-42c36b8a0740 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:24.661414 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:24.707772 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae5a74-14d4-4bd3-81a1-6e983260d8db None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:24.708105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae5a74-14d4-4bd3-81a1-6e983260d8db None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:24.708344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae5a74-14d4-4bd3-81a1-6e983260d8db None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:24.710076 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 461/908] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:24.715946 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f41541-38c6-4e07-a4dc-7424565c7ae2 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:24.716290 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f41541-38c6-4e07-a4dc-7424565c7ae2 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:24.716498 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f41541-38c6-4e07-a4dc-7424565c7ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:24.733850 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-77f41541-38c6-4e07-a4dc-7424565c7ae2 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:24.734949 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-77f41541-38c6-4e07-a4dc-7424565c7ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:24.780004 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604e85e-8ae4-4396-b2b8-42c36b8a0740 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:24.814524 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 462/909] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 13:33:24.829974 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c8a65-96c8-4a56-a445-5bcff890024f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:24.830782 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c8a65-96c8-4a56-a445-5bcff890024f None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:24.831071 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c8a65-96c8-4a56-a445-5bcff890024f None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:24.831348 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c8a65-96c8-4a56-a445-5bcff890024f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:24.842597 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f41541-38c6-4e07-a4dc-7424565c7ae2 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:24.842787 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c8a65-96c8-4a56-a445-5bcff890024f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:24.844061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c8a65-96c8-4a56-a445-5bcff890024f None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:24.863209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-823c8a65-96c8-4a56-a445-5bcff890024f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:24.863209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:24.863209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:24.863209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:24.863209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:24.863209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:24.863209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:24.863209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:24.863209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:24.863209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:24.863209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:24.863209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:24.863209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:24.863209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:24.863209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:24.864145 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:24.864145 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:24.864145 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:24.864145 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:24.864145 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:24.864145 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:24.864145 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:24.864145 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:24.864145 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:24.864145 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:24.864145 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:24.864145 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:24.864145 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:24.864145 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:24.864145 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:24.864145 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:24.864145 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:24.865061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:24.865061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:24.865061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:24.865061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:24.869258 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 448/910] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:24.870928 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c8a65-96c8-4a56-a445-5bcff890024f None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:24.942364 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:24.952704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:24.963785 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f41541-38c6-4e07-a4dc-7424565c7ae2 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:25.002775 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c8a65-96c8-4a56-a445-5bcff890024f None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:25.005407 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad88f93-9407-4c7a-9b90-b5d5287aaa64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:25.007119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad88f93-9407-4c7a-9b90-b5d5287aaa64 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:25.007408 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad88f93-9407-4c7a-9b90-b5d5287aaa64 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:25.007689 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad88f93-9407-4c7a-9b90-b5d5287aaa64 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:25.008352 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad88f93-9407-4c7a-9b90-b5d5287aaa64 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:25.009782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad88f93-9407-4c7a-9b90-b5d5287aaa64 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:25.025729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-1ad88f93-9407-4c7a-9b90-b5d5287aaa64 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 06 13:33:25.025729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:25.025729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 13:33:25.025729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 13:33:25.025729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 13:33:25.025729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 13:33:25.025729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 13:33:25.025729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:25.025729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 06 13:33:25.025729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:25.025729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 13:33:25.025729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:25.025729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:25.025729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:25.025729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:25.025729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:25.025729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:25.026951 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:25.026951 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:25.026951 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:25.026951 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:25.026951 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:25.026951 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:25.026951 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 13:33:25.026951 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 13:33:25.026951 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 13:33:25.026951 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 13:33:25.026951 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:25.026951 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:25.026951 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:25.026951 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:25.026951 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:25.026951 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:25.027866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:25.027866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:25.027866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:25.027866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:25.027866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:25.027866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:25.027866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:25.027866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:25.027866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:25.027866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:25.027866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 13:33:25.027866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:25.027866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 06 13:33:25.027866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:25.032723 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 463/911] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jul 6 13:33:24 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:33:25.033594 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad88f93-9407-4c7a-9b90-b5d5287aaa64 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:25.048834 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe7ce11-27b9-4702-9e6d-335575049772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:25.049824 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7ce11-27b9-4702-9e6d-335575049772 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:25.050290 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7ce11-27b9-4702-9e6d-335575049772 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:25.050745 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7ce11-27b9-4702-9e6d-335575049772 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:25.051654 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe7ce11-27b9-4702-9e6d-335575049772 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:25.052158 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe7ce11-27b9-4702-9e6d-335575049772 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:25.053806 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe7ce11-27b9-4702-9e6d-335575049772 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:25.070941 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 449/912] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:25.072594 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe7ce11-27b9-4702-9e6d-335575049772 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:25.155986 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad88f93-9407-4c7a-9b90-b5d5287aaa64 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:25.179805 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe7ce11-27b9-4702-9e6d-335575049772 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:25.193439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be87eb3-31fa-407d-baa7-22e6572e55f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:25.194201 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be87eb3-31fa-407d-baa7-22e6572e55f2 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:25.194490 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be87eb3-31fa-407d-baa7-22e6572e55f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:25.194770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be87eb3-31fa-407d-baa7-22e6572e55f2 None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:25.195495 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:25.196321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be87eb3-31fa-407d-baa7-22e6572e55f2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:25.197522 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be87eb3-31fa-407d-baa7-22e6572e55f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:25.199150 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be87eb3-31fa-407d-baa7-22e6572e55f2 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:25.210128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 464/913] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jul 6 13:33:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:25.211296 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be87eb3-31fa-407d-baa7-22e6572e55f2 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:25.219879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf692fce-112a-4f6c-9260-7ad04e6734d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:25.220764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf692fce-112a-4f6c-9260-7ad04e6734d4 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:25.221171 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf692fce-112a-4f6c-9260-7ad04e6734d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:25.221521 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf692fce-112a-4f6c-9260-7ad04e6734d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:25.228710 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf692fce-112a-4f6c-9260-7ad04e6734d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:25.230474 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf692fce-112a-4f6c-9260-7ad04e6734d4 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:25.238119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:25.246777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-cf692fce-112a-4f6c-9260-7ad04e6734d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:25.246777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:25.246777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:25.246777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:25.246777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:25.246777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:25.246777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:25.246777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:25.246777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:25.246777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:25.246777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:25.246777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:25.246777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:33:25.246777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:33:25.246777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:33:25.247916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:33:25.247916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:25.247916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:25.247916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:25.247916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:25.247916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:25.247916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:25.247916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:25.247916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:25.247916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:25.247916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:25.247916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:25.247916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:25.247916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:25.247916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:25.247916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:25.247916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:25.249066 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:33:25.249066 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:33:25.249066 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:33:25.249066 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:33:25.249066 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:33:25.249066 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:33:25.249066 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:25.249066 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:25.254877 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 450/914] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:33:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:33:25.256041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf692fce-112a-4f6c-9260-7ad04e6734d4 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:25.333151 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be87eb3-31fa-407d-baa7-22e6572e55f2 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:25.366940 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a276a6e6-de3f-4f84-b099-1b813d5e2169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:25.367602 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276a6e6-de3f-4f84-b099-1b813d5e2169 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:25.367705 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf692fce-112a-4f6c-9260-7ad04e6734d4 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:25.367885 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276a6e6-de3f-4f84-b099-1b813d5e2169 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:25.368161 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276a6e6-de3f-4f84-b099-1b813d5e2169 None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:25.369050 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a276a6e6-de3f-4f84-b099-1b813d5e2169 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:25.369390 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a276a6e6-de3f-4f84-b099-1b813d5e2169 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:25.370811 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a276a6e6-de3f-4f84-b099-1b813d5e2169 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:25.375834 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 465/915] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jul 6 13:33:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:25.377165 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3aef0-f61a-4476-9480-85c636de38ac None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:25.377451 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3aef0-f61a-4476-9480-85c636de38ac None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:25.377726 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3aef0-f61a-4476-9480-85c636de38ac None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:25.380949 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 466/916] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:25.381136 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3aef0-f61a-4476-9480-85c636de38ac None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:25.410884 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6425fe9b-28b4-475c-975a-7550ccff905f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:25.411543 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425fe9b-28b4-475c-975a-7550ccff905f None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:25.411825 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425fe9b-28b4-475c-975a-7550ccff905f None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:25.412099 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425fe9b-28b4-475c-975a-7550ccff905f None admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:25.412886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6425fe9b-28b4-475c-975a-7550ccff905f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:25.413220 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6425fe9b-28b4-475c-975a-7550ccff905f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:25.414706 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6425fe9b-28b4-475c-975a-7550ccff905f None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:25.427460 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 451/917] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:33:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:25.428509 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08d9e6-17e7-45c4-b47a-9a19ad82f6d4 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:25.428798 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08d9e6-17e7-45c4-b47a-9a19ad82f6d4 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:25.429104 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08d9e6-17e7-45c4-b47a-9a19ad82f6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:25.446003 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-cf08d9e6-17e7-45c4-b47a-9a19ad82f6d4 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:25.447741 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-cf08d9e6-17e7-45c4-b47a-9a19ad82f6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:25.475794 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:25.478441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:25.498270 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3aef0-f61a-4476-9480-85c636de38ac None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:25.519498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 452/918] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:25.520110 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08d9e6-17e7-45c4-b47a-9a19ad82f6d4 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:25.548303 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a4ccc-e7f4-4579-adcb-dfebaaa0816e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:25.549258 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a4ccc-e7f4-4579-adcb-dfebaaa0816e None admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:25.549537 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a4ccc-e7f4-4579-adcb-dfebaaa0816e None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:25.549809 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a4ccc-e7f4-4579-adcb-dfebaaa0816e None admin] PATH_INFO: `/v3/endpoints` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:25.550747 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a4ccc-e7f4-4579-adcb-dfebaaa0816e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:25.552101 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a4ccc-e7f4-4579-adcb-dfebaaa0816e None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:25.589037 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 467/919] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Jul 6 13:33:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:33:25.589037 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a4ccc-e7f4-4579-adcb-dfebaaa0816e None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:25.664986 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08d9e6-17e7-45c4-b47a-9a19ad82f6d4 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:25.714269 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a4ccc-e7f4-4579-adcb-dfebaaa0816e None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:25.715902 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45e487-e71c-480e-83d9-cc7bc23d7dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:25.716756 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45e487-e71c-480e-83d9-cc7bc23d7dcb None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:25.717971 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45e487-e71c-480e-83d9-cc7bc23d7dcb None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:25.718254 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45e487-e71c-480e-83d9-cc7bc23d7dcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:25.719114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e45e487-e71c-480e-83d9-cc7bc23d7dcb None admin] RBAC: Adding query filter params (role.id=8d259fab1ed64ced91f18487e90ee097, scope.project.id=bfe6c3d8da2743be9609184db2506303, user.id=c4535134181e489fb532f617264d5855) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:25.719427 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e45e487-e71c-480e-83d9-cc7bc23d7dcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:25.720884 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e45e487-e71c-480e-83d9-cc7bc23d7dcb None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:25.729971 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 453/920] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:33:25 2022] GET /identity/v3/role_assignments?user.id=c4535134181e489fb532f617264d5855&scope.project.id=bfe6c3d8da2743be9609184db2506303&role.id=8d259fab1ed64ced91f18487e90ee097 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:25.761215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65cda8-a104-4ad0-81c4-81011e91dbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:25.762043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65cda8-a104-4ad0-81c4-81011e91dbdd None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:25.762325 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65cda8-a104-4ad0-81c4-81011e91dbdd None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:25.762599 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65cda8-a104-4ad0-81c4-81011e91dbdd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:25.763269 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c65cda8-a104-4ad0-81c4-81011e91dbdd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:25.764660 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c65cda8-a104-4ad0-81c4-81011e91dbdd None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:25.767891 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:25.779544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-2c65cda8-a104-4ad0-81c4-81011e91dbdd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 13:33:25.779544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:25.779544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 13:33:25.779544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 13:33:25.779544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 13:33:25.779544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 13:33:25.779544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 13:33:25.779544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:25.779544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 13:33:25.779544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:25.779544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 13:33:25.779544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:25.779544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:25.779544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:25.779544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:25.779544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:25.779544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:25.780793 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:25.780793 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:25.780793 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:25.780793 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:25.780793 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:25.780793 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:25.780793 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 13:33:25.780793 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 13:33:25.780793 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 13:33:25.780793 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 13:33:25.780793 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:25.780793 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:25.780793 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:25.780793 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:25.780793 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:25.780793 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:25.781864 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:25.781864 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:25.781864 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:25.781864 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:25.781864 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:25.781864 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:25.781864 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:25.781864 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:25.781864 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:25.781864 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:25.781864 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 13:33:25.781864 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:25.781864 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 13:33:25.781864 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:25.785431 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 468/921] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Jul 6 13:33:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:25.791529 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45e487-e71c-480e-83d9-cc7bc23d7dcb None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:25.826880 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:25.909756 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45e487-e71c-480e-83d9-cc7bc23d7dcb None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:25.946551 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16768a88-f7b1-49b8-baa6-76ab845ec8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:25.948125 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16768a88-f7b1-49b8-baa6-76ab845ec8b6 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:25.948584 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16768a88-f7b1-49b8-baa6-76ab845ec8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:25.949060 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16768a88-f7b1-49b8-baa6-76ab845ec8b6 None admin] PATH_INFO: `/v3/services` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:25.949945 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16768a88-f7b1-49b8-baa6-76ab845ec8b6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:25.950426 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16768a88-f7b1-49b8-baa6-76ab845ec8b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:25.952002 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16768a88-f7b1-49b8-baa6-76ab845ec8b6 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:25.957691 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 454/922] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Jul 6 13:33:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:25.963850 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65cda8-a104-4ad0-81c4-81011e91dbdd None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:26.070553 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65cda8-a104-4ad0-81c4-81011e91dbdd None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:26.106233 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dfa082-c352-4603-a8d9-750d8c4f6f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:26.107236 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfa082-c352-4603-a8d9-750d8c4f6f16 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:26.107703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfa082-c352-4603-a8d9-750d8c4f6f16 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:26.108157 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfa082-c352-4603-a8d9-750d8c4f6f16 None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:26.109064 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dfa082-c352-4603-a8d9-750d8c4f6f16 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:26.109548 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dfa082-c352-4603-a8d9-750d8c4f6f16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:26.111100 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dfa082-c352-4603-a8d9-750d8c4f6f16 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:26.116847 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 469/923] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Jul 6 13:33:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:26.129057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16768a88-f7b1-49b8-baa6-76ab845ec8b6 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:26.226628 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:26.237404 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16768a88-f7b1-49b8-baa6-76ab845ec8b6 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:26.273861 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838b05f-41d1-4d63-bce6-37139d60ed82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:26.274526 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838b05f-41d1-4d63-bce6-37139d60ed82 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:26.275389 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838b05f-41d1-4d63-bce6-37139d60ed82 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:26.275737 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838b05f-41d1-4d63-bce6-37139d60ed82 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:26.277052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9838b05f-41d1-4d63-bce6-37139d60ed82 None admin] RBAC: Adding query filter params (interface=public, service_id=17625ac4b8a5490680fdf1d9ff84bffd, region_id=RegionOne) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:26.277375 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9838b05f-41d1-4d63-bce6-37139d60ed82 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:26.278751 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9838b05f-41d1-4d63-bce6-37139d60ed82 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:26.288526 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 455/924] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Jul 6 13:33:26 2022] GET /identity/v3/endpoints?service_id=17625ac4b8a5490680fdf1d9ff84bffd&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:26.289956 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dfa082-c352-4603-a8d9-750d8c4f6f16 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:26.360795 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:26.401729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dfa082-c352-4603-a8d9-750d8c4f6f16 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:26.437962 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374447f6-0784-4f56-98f3-567c14b3132f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:26.438687 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374447f6-0784-4f56-98f3-567c14b3132f None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:26.438978 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374447f6-0784-4f56-98f3-567c14b3132f None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:26.439255 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374447f6-0784-4f56-98f3-567c14b3132f None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:26.439944 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374447f6-0784-4f56-98f3-567c14b3132f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:26.441330 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374447f6-0784-4f56-98f3-567c14b3132f None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:26.453143 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 470/925] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Jul 6 13:33:26 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:33:26.463454 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3055a7-7330-4126-8d49-261d1f22a787 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:26.463794 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3055a7-7330-4126-8d49-261d1f22a787 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:26.464022 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3055a7-7330-4126-8d49-261d1f22a787 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:26.465827 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 456/926] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:26.471589 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14993c0-5d42-4f71-b6b4-ba54510f8dfd None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:26.471907 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14993c0-5d42-4f71-b6b4-ba54510f8dfd None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:26.472159 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14993c0-5d42-4f71-b6b4-ba54510f8dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:26.491026 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-d14993c0-5d42-4f71-b6b4-ba54510f8dfd None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:26.492230 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-d14993c0-5d42-4f71-b6b4-ba54510f8dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:26.565484 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 471/927] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:26.589312 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3055a7-7330-4126-8d49-261d1f22a787 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:26.699752 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3055a7-7330-4126-8d49-261d1f22a787 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:26.735534 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a21b8-4469-454c-8ff6-6c7403750a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:26.736332 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a21b8-4469-454c-8ff6-6c7403750a94 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:26.736621 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a21b8-4469-454c-8ff6-6c7403750a94 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:26.736898 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a21b8-4469-454c-8ff6-6c7403750a94 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:26.737585 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a21b8-4469-454c-8ff6-6c7403750a94 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:26.739064 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a21b8-4469-454c-8ff6-6c7403750a94 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:26.751685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-996a21b8-4469-454c-8ff6-6c7403750a94 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 06 13:33:26.751685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:26.751685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 13:33:26.751685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 13:33:26.751685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 13:33:26.751685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 13:33:26.751685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 13:33:26.751685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:26.751685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 06 13:33:26.751685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:26.751685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 13:33:26.751685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:26.751685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:26.751685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:26.751685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:26.751685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:26.752617 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:26.752617 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:26.752617 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:26.752617 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:26.752617 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:26.752617 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:26.752617 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:26.752617 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 13:33:26.752617 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 13:33:26.752617 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 13:33:26.752617 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 13:33:26.752617 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:26.752617 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:26.752617 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:26.752617 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:26.752617 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:26.753488 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:26.753488 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:26.753488 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:26.753488 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:26.753488 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:26.753488 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:26.753488 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:26.753488 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:26.753488 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:26.753488 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:26.753488 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:26.753488 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 13:33:26.753488 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:26.753488 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 06 13:33:26.753488 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:26.753488 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 457/928] 10.222.0.20 () {56 vars in 1132 bytes} [Wed Jul 6 13:33:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:33:26.759683 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14993c0-5d42-4f71-b6b4-ba54510f8dfd None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:26.820131 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:26.866464 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14993c0-5d42-4f71-b6b4-ba54510f8dfd None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:26.909080 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b2aee-1d1a-4068-8e0a-0109d8d007a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:26.909080 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b2aee-1d1a-4068-8e0a-0109d8d007a1 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:26.909080 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b2aee-1d1a-4068-8e0a-0109d8d007a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:26.909080 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b2aee-1d1a-4068-8e0a-0109d8d007a1 None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:26.909511 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7b2aee-1d1a-4068-8e0a-0109d8d007a1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:26.909511 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7b2aee-1d1a-4068-8e0a-0109d8d007a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:26.909511 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7b2aee-1d1a-4068-8e0a-0109d8d007a1 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:26.913024 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 472/929] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Jul 6 13:33:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:26.918415 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a21b8-4469-454c-8ff6-6c7403750a94 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:26.947328 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:27.050692 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a21b8-4469-454c-8ff6-6c7403750a94 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:27.088649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ca6f1-33ce-4343-8fc0-7b79fc504edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:27.089405 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ca6f1-33ce-4343-8fc0-7b79fc504edf None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:27.089690 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ca6f1-33ce-4343-8fc0-7b79fc504edf None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:27.089966 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ca6f1-33ce-4343-8fc0-7b79fc504edf None admin] PATH_INFO: `/v3/services` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:27.090688 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2ca6f1-33ce-4343-8fc0-7b79fc504edf None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:27.090996 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2ca6f1-33ce-4343-8fc0-7b79fc504edf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:27.092377 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2ca6f1-33ce-4343-8fc0-7b79fc504edf None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:27.098119 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 458/930] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Jul 6 13:33:26 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:27.104511 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b2aee-1d1a-4068-8e0a-0109d8d007a1 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:27.207951 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b2aee-1d1a-4068-8e0a-0109d8d007a1 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:27.230800 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:27.241845 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ecaa58-9e22-4ea6-af05-a0c3ab3e6ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:27.242512 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ecaa58-9e22-4ea6-af05-a0c3ab3e6ecf None admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:27.242792 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ecaa58-9e22-4ea6-af05-a0c3ab3e6ecf None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:27.243063 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ecaa58-9e22-4ea6-af05-a0c3ab3e6ecf None admin] PATH_INFO: `/v3/endpoints` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:27.244474 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ecaa58-9e22-4ea6-af05-a0c3ab3e6ecf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:27.245782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ecaa58-9e22-4ea6-af05-a0c3ab3e6ecf None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:27.268003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 473/931] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Jul 6 13:33:27 2022] POST /identity/v3/endpoints => generated 347 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:33:27.827793 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d0f3e-f0e7-42e7-8650-4d45cadff46e None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:27.828358 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d0f3e-f0e7-42e7-8650-4d45cadff46e None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:27.828441 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d0f3e-f0e7-42e7-8650-4d45cadff46e None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:27.829948 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 459/932] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:27.835332 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973485d7-d119-49f1-a74f-8b363f9f2c6a None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:27.835870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973485d7-d119-49f1-a74f-8b363f9f2c6a None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:27.836366 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973485d7-d119-49f1-a74f-8b363f9f2c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:27.855228 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-973485d7-d119-49f1-a74f-8b363f9f2c6a None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:27.856768 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-973485d7-d119-49f1-a74f-8b363f9f2c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:27.937570 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ef8be-dd6c-4dac-b8c3-493e2c3c5060 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:27.939051 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ef8be-dd6c-4dac-b8c3-493e2c3c5060 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:27.939051 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ef8be-dd6c-4dac-b8c3-493e2c3c5060 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:27.939426 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 474/933] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:27.940291 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 460/934] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:27.946619 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e8c0d-a39e-4be1-a33d-b78bbbaab31c None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:27.946977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e8c0d-a39e-4be1-a33d-b78bbbaab31c None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:27.947374 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e8c0d-a39e-4be1-a33d-b78bbbaab31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:27.964619 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-488e8c0d-a39e-4be1-a33d-b78bbbaab31c None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:27.965718 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-488e8c0d-a39e-4be1-a33d-b78bbbaab31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:27.966159 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ef8be-dd6c-4dac-b8c3-493e2c3c5060 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:28.049113 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 475/935] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 13:33:28.063893 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e8c0d-a39e-4be1-a33d-b78bbbaab31c None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:28.072051 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:28.097425 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ef8be-dd6c-4dac-b8c3-493e2c3c5060 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:28.102558 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:28.145770 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ee0b0-1c6b-4d3a-b03e-e3962a20da19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:28.146459 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ee0b0-1c6b-4d3a-b03e-e3962a20da19 None admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:28.146750 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ee0b0-1c6b-4d3a-b03e-e3962a20da19 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:28.147030 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ee0b0-1c6b-4d3a-b03e-e3962a20da19 None admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:28.147752 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ee0b0-1c6b-4d3a-b03e-e3962a20da19 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:28.149213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ee0b0-1c6b-4d3a-b03e-e3962a20da19 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:28.150619 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: WARNING keystone.server.flask.common [None req-937ee0b0-1c6b-4d3a-b03e-e3962a20da19 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 06 13:33:28.180682 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.sql.core [None req-937ee0b0-1c6b-4d3a-b03e-e3962a20da19 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 06 13:33:28.180682 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 06 13:33:28.180682 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [parameters: {'id': '52915da4f3354d1ebbd1557227c97097', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 06 13:33:28.180682 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86850) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 06 13:33:28.194918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-937ee0b0-1c6b-4d3a-b03e-e3962a20da19 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 52915da4f3354d1ebbd1557227c97097.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 52915da4f3354d1ebbd1557227c97097. Jul 06 13:33:28.194918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:28.194918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 06 13:33:28.194918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 06 13:33:28.194918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 06 13:33:28.194918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 06 13:33:28.194918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 06 13:33:28.194918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application result = self._query(query) Jul 06 13:33:28.194918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 06 13:33:28.194918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application conn.query(q) Jul 06 13:33:28.194918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 06 13:33:28.194918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 06 13:33:28.194918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 06 13:33:28.195817 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application result.read() Jul 06 13:33:28.195817 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 06 13:33:28.195817 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 06 13:33:28.195817 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 06 13:33:28.195817 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 06 13:33:28.195817 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 06 13:33:28.195817 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 06 13:33:28.195817 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 06 13:33:28.195817 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 06 13:33:28.195817 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 06 13:33:28.195817 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:28.195817 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 06 13:33:28.195817 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:28.195817 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:28.195817 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 06 13:33:28.195817 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 06 13:33:28.195817 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 06 13:33:28.195817 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 06 13:33:28.196803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 06 13:33:28.196803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application next(self.gen) Jul 06 13:33:28.196803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 06 13:33:28.196803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application yield resource Jul 06 13:33:28.196803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 06 13:33:28.196803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application next(self.gen) Jul 06 13:33:28.196803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 06 13:33:28.196803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self.session.rollback() Jul 06 13:33:28.196803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 06 13:33:28.196803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self.force_reraise() Jul 06 13:33:28.196803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 06 13:33:28.196803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise self.value Jul 06 13:33:28.196803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 06 13:33:28.196803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 06 13:33:28.196803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 06 13:33:28.196803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application session.commit() Jul 06 13:33:28.196803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 06 13:33:28.197737 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 06 13:33:28.197737 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 06 13:33:28.197737 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._prepare_impl() Jul 06 13:33:28.197737 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 06 13:33:28.197737 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self.session.flush() Jul 06 13:33:28.197737 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 06 13:33:28.197737 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._flush(objects) Jul 06 13:33:28.197737 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 06 13:33:28.197737 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 06 13:33:28.197737 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 06 13:33:28.197737 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application compat.raise_( Jul 06 13:33:28.197737 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 06 13:33:28.197737 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception Jul 06 13:33:28.197737 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 06 13:33:28.197737 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application flush_context.execute() Jul 06 13:33:28.197737 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 06 13:33:28.197737 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rec.execute(self) Jul 06 13:33:28.198708 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 06 13:33:28.198708 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 06 13:33:28.198708 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 06 13:33:28.198708 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 06 13:33:28.198708 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 06 13:33:28.198708 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 06 13:33:28.198708 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 06 13:33:28.198708 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 06 13:33:28.198708 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 06 13:33:28.198708 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 06 13:33:28.198708 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 06 13:33:28.198708 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 06 13:33:28.198708 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 06 13:33:28.198708 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 06 13:33:28.198708 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 06 13:33:28.199504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 06 13:33:28.199504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 06 13:33:28.199504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception Jul 06 13:33:28.199504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 06 13:33:28.199504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 06 13:33:28.199504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 06 13:33:28.199504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 06 13:33:28.199504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 06 13:33:28.199504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application result = self._query(query) Jul 06 13:33:28.199504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 06 13:33:28.199504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application conn.query(q) Jul 06 13:33:28.199504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 06 13:33:28.199504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 06 13:33:28.199504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 06 13:33:28.199504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application result.read() Jul 06 13:33:28.199504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 06 13:33:28.199504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 06 13:33:28.200362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 06 13:33:28.200362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 06 13:33:28.200362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 06 13:33:28.200362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 06 13:33:28.200362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 06 13:33:28.200362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 06 13:33:28.200362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 06 13:33:28.200362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 06 13:33:28.200362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [parameters: {'id': '52915da4f3354d1ebbd1557227c97097', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 06 13:33:28.200362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 06 13:33:28.200362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:28.200362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 13:33:28.200362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:28.200362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:28.200362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 06 13:33:28.201373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 06 13:33:28.201373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 06 13:33:28.201373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 06 13:33:28.201373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 06 13:33:28.201373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:28.201373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 13:33:28.201373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:28.201373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:28.201373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:28.201373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:28.201373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:28.201373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:28.201373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:28.201373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:28.201373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:28.201373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:28.201373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:28.202696 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:28.202696 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 06 13:33:28.202696 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 06 13:33:28.202696 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:28.202696 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:28.202696 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 06 13:33:28.202696 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 06 13:33:28.202696 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 52915da4f3354d1ebbd1557227c97097. Jul 06 13:33:28.202696 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:28.203732 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e8c0d-a39e-4be1-a33d-b78bbbaab31c None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:28.206066 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 461/936] 10.222.0.20 () {60 vars in 1140 bytes} [Wed Jul 6 13:33:27 2022] POST /identity/v3/projects => generated 262 bytes in 240 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:28.212187 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ee0b0-1c6b-4d3a-b03e-e3962a20da19 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:28.240332 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1466f02-a467-4f3c-81d3-0b1fb9bb224d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:28.241591 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1466f02-a467-4f3c-81d3-0b1fb9bb224d None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:28.242080 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1466f02-a467-4f3c-81d3-0b1fb9bb224d None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:28.242547 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1466f02-a467-4f3c-81d3-0b1fb9bb224d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:28.255616 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1466f02-a467-4f3c-81d3-0b1fb9bb224d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:33:28.264921 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-d1466f02-a467-4f3c-81d3-0b1fb9bb224d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 13:33:28.264921 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:28.264921 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:28.264921 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:28.264921 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:28.264921 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:28.264921 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:28.264921 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:28.264921 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:28.264921 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:28.264921 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:28.264921 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:28.264921 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:33:28.264921 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:33:28.264921 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:33:28.264921 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:33:28.265931 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:33:28.265931 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:33:28.265931 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:33:28.265931 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:33:28.265931 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:28.265931 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:28.265931 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:28.265931 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:28.265931 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:28.265931 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:28.265931 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:28.265931 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:28.265931 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:28.265931 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:28.265931 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:28.265931 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:28.266785 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:28.266785 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:28.266785 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:28.266785 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:28.266785 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:33:28.266785 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:33:28.266785 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:33:28.266785 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:33:28.266785 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:33:28.266785 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:33:28.266785 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 13:33:28.266785 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:28.274049 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 476/937] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Jul 6 13:33:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:33:28.275544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1466f02-a467-4f3c-81d3-0b1fb9bb224d None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:28.332692 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ee0b0-1c6b-4d3a-b03e-e3962a20da19 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:28.368396 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b360c-7c00-467b-883a-ac49f560a98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:28.369762 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b360c-7c00-467b-883a-ac49f560a98d None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:28.370382 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b360c-7c00-467b-883a-ac49f560a98d None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:28.370972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b360c-7c00-467b-883a-ac49f560a98d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:28.380115 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b360c-7c00-467b-883a-ac49f560a98d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:28.381959 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b360c-7c00-467b-883a-ac49f560a98d None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:28.385846 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:28.388904 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:28.396147 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1466f02-a467-4f3c-81d3-0b1fb9bb224d None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:28.398126 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-af3b360c-7c00-467b-883a-ac49f560a98d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:28.398126 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:28.398126 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:28.398126 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:28.398126 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:28.398126 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:28.398126 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:28.398126 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:28.398126 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:28.398126 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:28.398126 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:28.398126 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:28.398126 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:33:28.398126 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:33:28.398126 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:33:28.399257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:33:28.399257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:28.399257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:28.399257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:28.399257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:28.399257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:28.399257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:28.399257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:28.399257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:28.399257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:28.399257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:28.399257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:28.399257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:28.399257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:28.399257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:28.399257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:28.399257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:28.400856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:33:28.400856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:33:28.400856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:33:28.400856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:33:28.400856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:33:28.400856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:33:28.400856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:28.400856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:28.408535 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 462/938] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:33:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:28.410107 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b360c-7c00-467b-883a-ac49f560a98d None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:28.439388 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c4363-ecde-4f50-a9ac-2b4eee24e505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:28.440343 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c4363-ecde-4f50-a9ac-2b4eee24e505 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:28.440796 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c4363-ecde-4f50-a9ac-2b4eee24e505 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:28.441271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c4363-ecde-4f50-a9ac-2b4eee24e505 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:28.442157 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c4363-ecde-4f50-a9ac-2b4eee24e505 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:28.442621 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c4363-ecde-4f50-a9ac-2b4eee24e505 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:28.444229 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c4363-ecde-4f50-a9ac-2b4eee24e505 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:28.458384 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 477/939] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Jul 6 13:33:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:28.459858 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c4363-ecde-4f50-a9ac-2b4eee24e505 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:28.520365 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b360c-7c00-467b-883a-ac49f560a98d None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:28.558634 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4b74a-bfb1-4796-a312-e6d181fa63bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:28.559469 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4b74a-bfb1-4796-a312-e6d181fa63bc None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:28.559877 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4b74a-bfb1-4796-a312-e6d181fa63bc None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:28.560477 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4b74a-bfb1-4796-a312-e6d181fa63bc None admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:28.561162 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b4b74a-bfb1-4796-a312-e6d181fa63bc None admin] RBAC: Adding query filter params (name=service) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:28.561595 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b4b74a-bfb1-4796-a312-e6d181fa63bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:28.563116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b4b74a-bfb1-4796-a312-e6d181fa63bc None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:28.569881 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c4363-ecde-4f50-a9ac-2b4eee24e505 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:28.593310 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 463/940] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Jul 6 13:33:28 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:28.607698 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e829eb9-fb68-42b3-b40a-9804a337fedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:28.608794 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e829eb9-fb68-42b3-b40a-9804a337fedd None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:28.609268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e829eb9-fb68-42b3-b40a-9804a337fedd None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:28.609706 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e829eb9-fb68-42b3-b40a-9804a337fedd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:28.623525 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e829eb9-fb68-42b3-b40a-9804a337fedd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:28.625364 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e829eb9-fb68-42b3-b40a-9804a337fedd None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:28.645859 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-6e829eb9-fb68-42b3-b40a-9804a337fedd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:28.645859 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:28.645859 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:28.645859 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:28.645859 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:28.645859 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:28.645859 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:28.645859 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:28.645859 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:28.645859 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:28.645859 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:28.645859 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:28.645859 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:28.645859 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:28.645859 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:28.647003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:28.647003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:28.647003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:28.647003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:28.647003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:28.647003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:28.647003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:28.647003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:28.647003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:28.647003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:28.647003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:28.647003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:28.647003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:28.647003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:28.647003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:28.647003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:28.647003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:28.648116 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:28.648116 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:28.648116 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:28.648116 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:28.654587 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 478/941] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:33:28.656256 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4b74a-bfb1-4796-a312-e6d181fa63bc None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:28.728712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:28.772893 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4b74a-bfb1-4796-a312-e6d181fa63bc None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:28.799823 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fb8ac-fee6-45a8-9a04-965a51735df6 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:28.800105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fb8ac-fee6-45a8-9a04-965a51735df6 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:28.800380 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fb8ac-fee6-45a8-9a04-965a51735df6 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:28.802049 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 479/942] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:28.808026 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0371fde-f4fa-4043-85db-bad84eee8424 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:28.808364 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0371fde-f4fa-4043-85db-bad84eee8424 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:28.808642 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0371fde-f4fa-4043-85db-bad84eee8424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:28.810571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ce477-7328-46ae-8f5e-8543cf4ee493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:28.811342 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ce477-7328-46ae-8f5e-8543cf4ee493 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:28.811729 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ce477-7328-46ae-8f5e-8543cf4ee493 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:28.812106 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ce477-7328-46ae-8f5e-8543cf4ee493 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:28.812941 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ce477-7328-46ae-8f5e-8543cf4ee493 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:28.813536 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ce477-7328-46ae-8f5e-8543cf4ee493 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:28.815007 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ce477-7328-46ae-8f5e-8543cf4ee493 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:28.827901 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-f0371fde-f4fa-4043-85db-bad84eee8424 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:28.829103 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-f0371fde-f4fa-4043-85db-bad84eee8424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:28.829231 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 464/943] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:28.832061 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:28.835666 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ce477-7328-46ae-8f5e-8543cf4ee493 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:28.924359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 480/944] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 13:33:28.946160 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ce477-7328-46ae-8f5e-8543cf4ee493 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:28.949830 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0371fde-f4fa-4043-85db-bad84eee8424 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:28.968117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:28.987070 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d566d89f-159b-4e51-b609-85c8b19187bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:28.987896 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d566d89f-159b-4e51-b609-85c8b19187bb None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:28.988181 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d566d89f-159b-4e51-b609-85c8b19187bb None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:28.988458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d566d89f-159b-4e51-b609-85c8b19187bb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:28.997142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d566d89f-159b-4e51-b609-85c8b19187bb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:28.999010 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d566d89f-159b-4e51-b609-85c8b19187bb None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:29.014603 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-d566d89f-159b-4e51-b609-85c8b19187bb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 13:33:29.014603 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:29.014603 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:29.014603 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:29.014603 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:29.014603 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:29.014603 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:29.014603 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:29.014603 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:29.014603 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:29.014603 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:29.014603 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:29.014603 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:33:29.014603 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:33:29.014603 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:33:29.014603 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:33:29.015607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:29.015607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:29.015607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:33:29.015607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:29.015607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:33:29.015607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:29.015607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:29.015607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:29.015607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:29.015607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:29.015607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:29.015607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:29.015607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:29.015607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:29.015607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:29.015607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:29.015607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:29.015607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:29.016539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:29.016539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:29.016539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:33:29.016539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:33:29.016539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:33:29.016539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:33:29.016539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:33:29.016539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:33:29.016539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 13:33:29.016539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:29.016539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 465/945] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Jul 6 13:33:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:29.021784 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d566d89f-159b-4e51-b609-85c8b19187bb None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:29.069824 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0371fde-f4fa-4043-85db-bad84eee8424 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:29.116582 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9fda8-a33d-46be-b022-e91e840faaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:29.117557 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9fda8-a33d-46be-b022-e91e840faaf7 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:29.117915 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9fda8-a33d-46be-b022-e91e840faaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:29.118267 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9fda8-a33d-46be-b022-e91e840faaf7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:29.119340 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9fda8-a33d-46be-b022-e91e840faaf7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:29.120588 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9fda8-a33d-46be-b022-e91e840faaf7 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:29.138642 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-71c9fda8-a33d-46be-b022-e91e840faaf7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 13:33:29.138642 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:29.138642 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 13:33:29.138642 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 13:33:29.138642 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 13:33:29.138642 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 13:33:29.138642 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 13:33:29.138642 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:29.138642 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 13:33:29.138642 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:29.138642 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 13:33:29.138642 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:29.138642 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:29.138642 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:29.138642 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:29.138642 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:29.138642 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:29.139698 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:29.139698 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:29.139698 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:29.139698 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:29.139698 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:29.139698 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:29.139698 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 13:33:29.139698 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 13:33:29.139698 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 13:33:29.139698 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 13:33:29.139698 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:29.139698 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:29.139698 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:29.139698 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:29.139698 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:29.139698 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:29.140539 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:29.140539 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:29.140539 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:29.140539 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:29.140539 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:29.140539 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:29.140539 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:29.140539 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:29.140539 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:29.140539 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:29.140539 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 13:33:29.140539 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:29.140539 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 13:33:29.140539 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:29.140539 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 481/946] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Jul 6 13:33:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:33:29.147302 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9fda8-a33d-46be-b022-e91e840faaf7 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:29.151398 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:29.160965 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d566d89f-159b-4e51-b609-85c8b19187bb None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:29.202383 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a13bc-a031-46f2-9bbd-abf780faeac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:29.203434 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a13bc-a031-46f2-9bbd-abf780faeac9 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:29.203922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a13bc-a031-46f2-9bbd-abf780faeac9 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:29.204931 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a13bc-a031-46f2-9bbd-abf780faeac9 None admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:29.206089 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a13bc-a031-46f2-9bbd-abf780faeac9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:29.207071 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a13bc-a031-46f2-9bbd-abf780faeac9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:29.209445 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a13bc-a031-46f2-9bbd-abf780faeac9 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:29.235546 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 466/947] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Jul 6 13:33:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:29.240566 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a13bc-a031-46f2-9bbd-abf780faeac9 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:29.276150 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9fda8-a33d-46be-b022-e91e840faaf7 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:29.297717 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:29.326221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed325754-e4f2-4dc9-8370-e3bdbc2360fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:29.327238 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed325754-e4f2-4dc9-8370-e3bdbc2360fc None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:29.327751 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed325754-e4f2-4dc9-8370-e3bdbc2360fc None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:29.328240 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed325754-e4f2-4dc9-8370-e3bdbc2360fc None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:29.329173 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed325754-e4f2-4dc9-8370-e3bdbc2360fc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:29.329707 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed325754-e4f2-4dc9-8370-e3bdbc2360fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:29.331267 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed325754-e4f2-4dc9-8370-e3bdbc2360fc None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:29.337804 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 482/948] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Jul 6 13:33:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:29.349033 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed325754-e4f2-4dc9-8370-e3bdbc2360fc None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:29.376910 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a13bc-a031-46f2-9bbd-abf780faeac9 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:29.414205 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:29.418058 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c934d64-ccca-4540-a1b8-1f0dba5dc7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:29.419025 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c934d64-ccca-4540-a1b8-1f0dba5dc7ef None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:29.420082 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c934d64-ccca-4540-a1b8-1f0dba5dc7ef None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:29.420367 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c934d64-ccca-4540-a1b8-1f0dba5dc7ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:29.435726 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c934d64-ccca-4540-a1b8-1f0dba5dc7ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:29.437224 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c934d64-ccca-4540-a1b8-1f0dba5dc7ef None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:29.451445 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed325754-e4f2-4dc9-8370-e3bdbc2360fc None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:29.458192 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-6c934d64-ccca-4540-a1b8-1f0dba5dc7ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:29.458192 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:29.458192 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:29.458192 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:29.458192 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:29.458192 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:29.458192 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:29.458192 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:29.458192 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:29.458192 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:29.458192 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:29.458192 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:29.458192 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:29.458192 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:29.458192 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:29.459229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:29.459229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:29.459229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:29.459229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:29.459229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:29.459229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:29.459229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:29.459229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:29.459229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:29.459229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:29.459229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:29.459229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:29.459229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:29.459229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:29.459229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:29.459229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:29.459229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:29.460196 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:29.460196 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:29.460196 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:29.460196 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:29.460196 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 467/949] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:33:29.461157 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23031cb8-c7ee-4f8a-8f17-3dcdbf664902 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:29.461446 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23031cb8-c7ee-4f8a-8f17-3dcdbf664902 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:29.461725 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23031cb8-c7ee-4f8a-8f17-3dcdbf664902 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:29.464378 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 468/950] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:29.466825 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23031cb8-c7ee-4f8a-8f17-3dcdbf664902 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:29.493221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f013cef-2a53-42af-ba44-7f7f61d70b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:29.495032 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f013cef-2a53-42af-ba44-7f7f61d70b75 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:29.495325 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f013cef-2a53-42af-ba44-7f7f61d70b75 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:29.495598 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f013cef-2a53-42af-ba44-7f7f61d70b75 None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:29.496307 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f013cef-2a53-42af-ba44-7f7f61d70b75 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:29.496618 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f013cef-2a53-42af-ba44-7f7f61d70b75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:29.498260 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f013cef-2a53-42af-ba44-7f7f61d70b75 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:29.508624 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 483/951] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Jul 6 13:33:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:29.510888 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4efc42-9405-4da4-b6e3-40b48428f991 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:29.511183 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4efc42-9405-4da4-b6e3-40b48428f991 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:29.511464 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4efc42-9405-4da4-b6e3-40b48428f991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:29.528501 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-1a4efc42-9405-4da4-b6e3-40b48428f991 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:29.529705 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-1a4efc42-9405-4da4-b6e3-40b48428f991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:29.563910 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:29.577753 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23031cb8-c7ee-4f8a-8f17-3dcdbf664902 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:29.605104 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 484/952] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:29.605104 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4efc42-9405-4da4-b6e3-40b48428f991 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:29.621659 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a319aa-2478-474e-b13f-ff636088921e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:29.622334 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a319aa-2478-474e-b13f-ff636088921e None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:29.622615 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a319aa-2478-474e-b13f-ff636088921e None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:29.622899 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a319aa-2478-474e-b13f-ff636088921e None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:29.623620 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a319aa-2478-474e-b13f-ff636088921e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:29.623932 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a319aa-2478-474e-b13f-ff636088921e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:29.625591 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a319aa-2478-474e-b13f-ff636088921e None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:29.642670 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 469/953] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:29.642670 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a319aa-2478-474e-b13f-ff636088921e None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:29.676649 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:29.720622 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4efc42-9405-4da4-b6e3-40b48428f991 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:29.749781 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a319aa-2478-474e-b13f-ff636088921e None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:29.756115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d19824-aac0-46f0-8c90-4317f10b4673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:29.756897 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d19824-aac0-46f0-8c90-4317f10b4673 None admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:29.757197 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d19824-aac0-46f0-8c90-4317f10b4673 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:29.757474 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d19824-aac0-46f0-8c90-4317f10b4673 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:29.758210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d19824-aac0-46f0-8c90-4317f10b4673 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:29.760214 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d19824-aac0-46f0-8c90-4317f10b4673 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:29.793041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 485/954] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Jul 6 13:33:29 2022] POST /identity/v3/endpoints => generated 349 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:33:29.793041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d19824-aac0-46f0-8c90-4317f10b4673 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:29.803729 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c9cf12-770e-4243-9579-524260061c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:29.804727 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9cf12-770e-4243-9579-524260061c1c None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:29.805144 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9cf12-770e-4243-9579-524260061c1c None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:29.805502 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9cf12-770e-4243-9579-524260061c1c None admin] PATH_INFO: `/v3/services/compute` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:29.806481 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c9cf12-770e-4243-9579-524260061c1c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:29.808162 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c9cf12-770e-4243-9579-524260061c1c None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:29.822885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-32c9cf12-770e-4243-9579-524260061c1c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 06 13:33:29.822885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:29.822885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 13:33:29.822885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 13:33:29.822885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 13:33:29.822885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 13:33:29.822885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 13:33:29.822885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:29.822885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 06 13:33:29.822885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:29.822885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 13:33:29.822885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:29.822885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:29.822885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:29.822885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:29.822885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:29.822885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:29.824098 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:29.824098 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:29.824098 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:29.824098 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:29.824098 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:29.824098 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:29.824098 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 13:33:29.824098 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 13:33:29.824098 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 13:33:29.824098 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 13:33:29.824098 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:29.824098 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:29.824098 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:29.824098 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:29.824098 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:29.824098 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:29.824988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:29.824988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:29.824988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:29.824988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:29.824988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:29.824988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:29.824988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:29.824988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:29.824988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:29.824988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:29.824988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 13:33:29.824988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:29.824988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 06 13:33:29.824988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:29.824988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 470/955] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:33:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:29.830251 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c9cf12-770e-4243-9579-524260061c1c None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:29.852305 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:29.925783 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d19824-aac0-46f0-8c90-4317f10b4673 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:29.949643 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:29.978598 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d412a-0446-4b62-8dbe-2d34e747e4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:29.979327 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d412a-0446-4b62-8dbe-2d34e747e4db None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:29.980193 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d412a-0446-4b62-8dbe-2d34e747e4db None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:29.980193 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d412a-0446-4b62-8dbe-2d34e747e4db None admin] PATH_INFO: `/v3/projects/service` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:29.983549 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c9cf12-770e-4243-9579-524260061c1c None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:29.993549 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d412a-0446-4b62-8dbe-2d34e747e4db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:29.995447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d412a-0446-4b62-8dbe-2d34e747e4db None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:30.011612 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-b35d412a-0446-4b62-8dbe-2d34e747e4db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:30.011612 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:30.011612 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:30.011612 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:30.011612 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:30.011612 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:30.011612 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:30.011612 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:30.011612 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:30.011612 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:30.011612 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:30.011612 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:30.011612 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:33:30.011612 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:33:30.011612 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:33:30.012592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:33:30.012592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:30.012592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:30.012592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:30.012592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:30.012592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:30.012592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:30.012592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:30.012592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:30.012592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:30.012592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:30.012592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:30.012592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:30.012592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:30.012592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:30.012592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:30.012592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:30.013584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:33:30.013584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:33:30.013584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:33:30.013584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:33:30.013584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:33:30.013584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:33:30.013584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:30.013584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:30.016141 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 486/956] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:33:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:30.021626 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d412a-0446-4b62-8dbe-2d34e747e4db None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:30.022134 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799486ab-4eb6-42ef-a8ab-3984f3206225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:30.022823 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799486ab-4eb6-42ef-a8ab-3984f3206225 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:30.023087 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799486ab-4eb6-42ef-a8ab-3984f3206225 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:30.023369 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799486ab-4eb6-42ef-a8ab-3984f3206225 None admin] PATH_INFO: `/v3/services` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:30.024113 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799486ab-4eb6-42ef-a8ab-3984f3206225 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:30.024420 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799486ab-4eb6-42ef-a8ab-3984f3206225 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:30.026051 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799486ab-4eb6-42ef-a8ab-3984f3206225 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:30.031648 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 471/957] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Jul 6 13:33:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:30.037107 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799486ab-4eb6-42ef-a8ab-3984f3206225 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:30.114690 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:30.140308 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799486ab-4eb6-42ef-a8ab-3984f3206225 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:30.141260 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d412a-0446-4b62-8dbe-2d34e747e4db None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:30.176638 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6c5317-66d9-4235-9db0-4cbd042cde68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:30.177360 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c5317-66d9-4235-9db0-4cbd042cde68 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:30.177609 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c5317-66d9-4235-9db0-4cbd042cde68 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:30.177885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c5317-66d9-4235-9db0-4cbd042cde68 None admin] PATH_INFO: `/v3/services` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:30.178593 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6c5317-66d9-4235-9db0-4cbd042cde68 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:30.178901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6c5317-66d9-4235-9db0-4cbd042cde68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:30.180260 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6c5317-66d9-4235-9db0-4cbd042cde68 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:30.182476 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1478ee8-ec06-4d92-9610-112748a7eab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:30.183482 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1478ee8-ec06-4d92-9610-112748a7eab0 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:30.183970 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1478ee8-ec06-4d92-9610-112748a7eab0 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:30.184448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1478ee8-ec06-4d92-9610-112748a7eab0 None admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:30.185608 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1478ee8-ec06-4d92-9610-112748a7eab0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:30.186110 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 472/958] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Jul 6 13:33:30 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:30.186290 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1478ee8-ec06-4d92-9610-112748a7eab0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:30.187773 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1478ee8-ec06-4d92-9610-112748a7eab0 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:30.202305 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 487/959] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:33:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:30.208260 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6c5317-66d9-4235-9db0-4cbd042cde68 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:30.325526 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6c5317-66d9-4235-9db0-4cbd042cde68 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:30.361599 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834c3e5-5f49-4dca-8a41-7447f8e4808d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:30.362489 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834c3e5-5f49-4dca-8a41-7447f8e4808d None admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:30.362772 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834c3e5-5f49-4dca-8a41-7447f8e4808d None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:30.363046 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834c3e5-5f49-4dca-8a41-7447f8e4808d None admin] PATH_INFO: `/v3/projects/bfe6c3d8da2743be9609184db2506303/users/c4535134181e489fb532f617264d5855/roles/8d259fab1ed64ced91f18487e90ee097` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:30.365274 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4834c3e5-5f49-4dca-8a41-7447f8e4808d None admin] RBAC: Authorizing `identity:create_grant(project_id=bfe6c3d8da2743be9609184db2506303, user_id=c4535134181e489fb532f617264d5855, role_id=8d259fab1ed64ced91f18487e90ee097)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:30.366701 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4834c3e5-5f49-4dca-8a41-7447f8e4808d None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:30.377507 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:30.383825 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 473/960] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jul 6 13:33:30 2022] PUT /identity/v3/projects/bfe6c3d8da2743be9609184db2506303/users/c4535134181e489fb532f617264d5855/roles/8d259fab1ed64ced91f18487e90ee097 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:33:30.772204 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570a676-9cd4-4ee1-a0cf-dffa21c74ba5 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:30.773097 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570a676-9cd4-4ee1-a0cf-dffa21c74ba5 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:30.773524 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570a676-9cd4-4ee1-a0cf-dffa21c74ba5 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:30.775833 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 488/961] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:30.781355 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531ffa4-4db0-4632-bc8b-2137d781287b None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:30.781806 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531ffa4-4db0-4632-bc8b-2137d781287b None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:30.782230 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531ffa4-4db0-4632-bc8b-2137d781287b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:30.799904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-a531ffa4-4db0-4632-bc8b-2137d781287b None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:30.801459 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a531ffa4-4db0-4632-bc8b-2137d781287b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:30.874411 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 474/962] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 13:33:30.901149 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570a676-9cd4-4ee1-a0cf-dffa21c74ba5 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:30.955570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:31.031202 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570a676-9cd4-4ee1-a0cf-dffa21c74ba5 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:31.069926 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a4ec4-94e2-4635-be9e-3a27ebf26a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:31.070600 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a4ec4-94e2-4635-be9e-3a27ebf26a51 None admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:31.070887 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a4ec4-94e2-4635-be9e-3a27ebf26a51 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:31.071164 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a4ec4-94e2-4635-be9e-3a27ebf26a51 None admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:31.072698 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22a4ec4-94e2-4635-be9e-3a27ebf26a51 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:31.074595 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22a4ec4-94e2-4635-be9e-3a27ebf26a51 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:31.075919 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: WARNING keystone.server.flask.common [None req-b22a4ec4-94e2-4635-be9e-3a27ebf26a51 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 06 13:33:31.092487 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.sql.core [None req-b22a4ec4-94e2-4635-be9e-3a27ebf26a51 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 06 13:33:31.092487 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 06 13:33:31.092487 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [parameters: {'user_id': '5ee6687520024567bd5f4e00852659ec', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 06 13:33:31.092487 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86849) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 06 13:33:31.114185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-b22a4ec4-94e2-4635-be9e-3a27ebf26a51 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 06 13:33:31.114185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:31.114185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 06 13:33:31.114185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 06 13:33:31.114185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 06 13:33:31.114185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 06 13:33:31.114185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 06 13:33:31.114185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application result = self._query(query) Jul 06 13:33:31.114185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 06 13:33:31.114185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application conn.query(q) Jul 06 13:33:31.114185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 06 13:33:31.114185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 06 13:33:31.114185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 06 13:33:31.114185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application result.read() Jul 06 13:33:31.114185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 06 13:33:31.115145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 06 13:33:31.115145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 06 13:33:31.115145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 06 13:33:31.115145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 06 13:33:31.115145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 06 13:33:31.115145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 06 13:33:31.115145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 06 13:33:31.115145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 06 13:33:31.115145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:31.115145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 06 13:33:31.115145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:31.115145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:31.115145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 06 13:33:31.115145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 06 13:33:31.115145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 06 13:33:31.115145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 06 13:33:31.115145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 06 13:33:31.115145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application next(self.gen) Jul 06 13:33:31.116103 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 06 13:33:31.116103 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application yield resource Jul 06 13:33:31.116103 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 06 13:33:31.116103 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application next(self.gen) Jul 06 13:33:31.116103 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 06 13:33:31.116103 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application self.session.rollback() Jul 06 13:33:31.116103 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 06 13:33:31.116103 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application self.force_reraise() Jul 06 13:33:31.116103 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 06 13:33:31.116103 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise self.value Jul 06 13:33:31.116103 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 06 13:33:31.116103 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 06 13:33:31.116103 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 06 13:33:31.116103 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application session.commit() Jul 06 13:33:31.116103 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 06 13:33:31.116103 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 06 13:33:31.116103 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 06 13:33:31.117584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application self._prepare_impl() Jul 06 13:33:31.117584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 06 13:33:31.117584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application self.session.flush() Jul 06 13:33:31.117584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 06 13:33:31.117584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application self._flush(objects) Jul 06 13:33:31.117584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 06 13:33:31.117584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 06 13:33:31.117584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 06 13:33:31.117584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application compat.raise_( Jul 06 13:33:31.117584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 06 13:33:31.117584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception Jul 06 13:33:31.117584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 06 13:33:31.117584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application flush_context.execute() Jul 06 13:33:31.117584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 06 13:33:31.117584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rec.execute(self) Jul 06 13:33:31.117584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 06 13:33:31.117584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 06 13:33:31.120318 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 06 13:33:31.120318 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 06 13:33:31.120318 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 06 13:33:31.120318 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 06 13:33:31.120318 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 06 13:33:31.120318 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 06 13:33:31.120318 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 06 13:33:31.120318 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 06 13:33:31.120318 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 06 13:33:31.120318 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 06 13:33:31.120318 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 06 13:33:31.120318 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 06 13:33:31.120318 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 06 13:33:31.120318 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 06 13:33:31.120318 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 06 13:33:31.120318 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception Jul 06 13:33:31.128667 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 06 13:33:31.128667 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 06 13:33:31.128667 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 06 13:33:31.128667 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 06 13:33:31.128667 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 06 13:33:31.128667 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application result = self._query(query) Jul 06 13:33:31.128667 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 06 13:33:31.128667 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application conn.query(q) Jul 06 13:33:31.128667 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 06 13:33:31.128667 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 06 13:33:31.128667 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 06 13:33:31.128667 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application result.read() Jul 06 13:33:31.128667 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 06 13:33:31.128667 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 06 13:33:31.128667 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 06 13:33:31.128667 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 06 13:33:31.130702 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 06 13:33:31.130702 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 06 13:33:31.130702 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 06 13:33:31.130702 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 06 13:33:31.130702 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 06 13:33:31.130702 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 06 13:33:31.130702 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [parameters: {'user_id': '5ee6687520024567bd5f4e00852659ec', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 06 13:33:31.130702 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 06 13:33:31.130702 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:31.130702 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 13:33:31.130702 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:31.130702 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:31.130702 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:31.130702 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:31.130702 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:31.132558 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:31.132558 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:31.132558 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:31.132558 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:31.132558 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:31.132558 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:31.132558 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:31.132558 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 06 13:33:31.132558 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 06 13:33:31.132558 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:31.132558 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:31.132558 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:33:31.132558 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:31.132558 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:33:31.132558 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:31.132558 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 06 13:33:31.132558 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 06 13:33:31.134978 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 06 13:33:31.134978 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 06 13:33:31.134978 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 06 13:33:31.134978 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 06 13:33:31.134978 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 06 13:33:31.134978 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:31.134978 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 489/963] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Jul 6 13:33:30 2022] POST /identity/v3/users => generated 160 bytes in 214 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:31.142157 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531ffa4-4db0-4632-bc8b-2137d781287b None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:31.310940 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531ffa4-4db0-4632-bc8b-2137d781287b None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:31.379649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:31.401724 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997191b7-a4ce-4c29-988d-42d3b02a7e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:31.402550 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997191b7-a4ce-4c29-988d-42d3b02a7e27 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:31.402835 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997191b7-a4ce-4c29-988d-42d3b02a7e27 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:31.403109 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997191b7-a4ce-4c29-988d-42d3b02a7e27 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:31.418110 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997191b7-a4ce-4c29-988d-42d3b02a7e27 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:31.419562 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997191b7-a4ce-4c29-988d-42d3b02a7e27 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:31.458204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-997191b7-a4ce-4c29-988d-42d3b02a7e27 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 06 13:33:31.458204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:31.458204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:31.458204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:31.458204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:31.458204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:31.458204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:31.458204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:31.458204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:31.458204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:31.458204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:31.458204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:31.458204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:33:31.458204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:33:31.458204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:33:31.458204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:33:31.459224 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:31.459224 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:31.459224 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:33:31.459224 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:31.459224 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:33:31.459224 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:31.459224 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:31.459224 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:31.459224 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:31.459224 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:31.459224 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:31.459224 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:31.459224 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:31.459224 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:31.459224 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:31.459224 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:31.459224 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:31.459224 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:31.465207 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:31.465207 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:31.465207 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:33:31.465207 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:33:31.465207 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:33:31.465207 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:33:31.465207 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:33:31.465207 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:33:31.465207 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 06 13:33:31.465207 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:31.465207 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 475/964] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:33:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:31.468757 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a4ec4-94e2-4635-be9e-3a27ebf26a51 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:31.641595 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:31.652593 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a4ec4-94e2-4635-be9e-3a27ebf26a51 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:31.693800 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7ae3f-32d6-447e-9270-e1184218e7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:31.694880 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7ae3f-32d6-447e-9270-e1184218e7ee None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:31.695405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7ae3f-32d6-447e-9270-e1184218e7ee None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:31.695890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7ae3f-32d6-447e-9270-e1184218e7ee None admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:31.697546 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7ae3f-32d6-447e-9270-e1184218e7ee None admin] RBAC: Adding query filter params (name=cinder) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:31.698151 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7ae3f-32d6-447e-9270-e1184218e7ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:31.700007 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7ae3f-32d6-447e-9270-e1184218e7ee None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:31.751882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 490/965] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jul 6 13:33:31 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:32.060642 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9560eb-4a4d-4742-8eb6-c002dfb0dcfd None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:32.061184 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9560eb-4a4d-4742-8eb6-c002dfb0dcfd None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:32.061263 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9560eb-4a4d-4742-8eb6-c002dfb0dcfd None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:32.062745 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 476/966] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:32.067898 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df70b0-fe0b-4a48-a85f-df413cd286a3 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:32.068323 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df70b0-fe0b-4a48-a85f-df413cd286a3 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:32.068485 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df70b0-fe0b-4a48-a85f-df413cd286a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:32.086187 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-84df70b0-fe0b-4a48-a85f-df413cd286a3 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:32.088255 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-84df70b0-fe0b-4a48-a85f-df413cd286a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:32.158013 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 491/967] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:32.181766 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9560eb-4a4d-4742-8eb6-c002dfb0dcfd None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:32.283451 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9560eb-4a4d-4742-8eb6-c002dfb0dcfd None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:32.324918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47acb59c-fdc7-4485-8e3b-887630220669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:32.325681 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47acb59c-fdc7-4485-8e3b-887630220669 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:32.325967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47acb59c-fdc7-4485-8e3b-887630220669 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:32.326294 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47acb59c-fdc7-4485-8e3b-887630220669 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:32.326959 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47acb59c-fdc7-4485-8e3b-887630220669 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:32.328321 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47acb59c-fdc7-4485-8e3b-887630220669 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:32.340524 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-47acb59c-fdc7-4485-8e3b-887630220669 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 13:33:32.340524 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:32.340524 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 13:33:32.340524 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 13:33:32.340524 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 13:33:32.340524 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 13:33:32.340524 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 13:33:32.340524 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:32.340524 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 13:33:32.340524 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:32.340524 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 13:33:32.340524 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:32.340524 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:32.340524 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:32.340524 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:32.340524 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:32.340524 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:32.341748 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:32.341748 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:32.341748 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:32.341748 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:32.341748 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:32.341748 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:32.341748 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 13:33:32.341748 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 13:33:32.341748 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 13:33:32.341748 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 13:33:32.341748 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:32.341748 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:32.341748 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:32.341748 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:32.341748 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:32.341748 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:32.342596 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:32.342596 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:32.342596 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:32.342596 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:32.342596 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:32.342596 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:32.342596 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:32.342596 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:32.342596 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:32.342596 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:32.342596 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 13:33:32.342596 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:32.342596 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 13:33:32.342596 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:32.342596 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 477/968] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Jul 6 13:33:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:32.349409 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df70b0-fe0b-4a48-a85f-df413cd286a3 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:32.439519 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:32.456134 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df70b0-fe0b-4a48-a85f-df413cd286a3 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:32.490877 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1b2c4-8bff-4cdf-a271-faa360e897b7 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:32.491220 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1b2c4-8bff-4cdf-a271-faa360e897b7 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:32.491334 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000bb29f-a137-4865-bd84-69d930f8faed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:32.491444 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1b2c4-8bff-4cdf-a271-faa360e897b7 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:32.491986 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bb29f-a137-4865-bd84-69d930f8faed None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:32.492271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bb29f-a137-4865-bd84-69d930f8faed None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:32.492548 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bb29f-a137-4865-bd84-69d930f8faed None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:32.496465 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 478/969] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:32.496610 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000bb29f-a137-4865-bd84-69d930f8faed None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:32.496610 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000bb29f-a137-4865-bd84-69d930f8faed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:32.496610 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000bb29f-a137-4865-bd84-69d930f8faed None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:32.497750 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91585728-8eff-46bb-8a09-97fccad8ce99 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:32.498034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91585728-8eff-46bb-8a09-97fccad8ce99 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:32.498310 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91585728-8eff-46bb-8a09-97fccad8ce99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:32.501435 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 492/970] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Jul 6 13:33:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:32.507750 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000bb29f-a137-4865-bd84-69d930f8faed None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:32.517724 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-91585728-8eff-46bb-8a09-97fccad8ce99 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:32.519267 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-91585728-8eff-46bb-8a09-97fccad8ce99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:32.569483 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:32.591499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 479/971] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:32.612006 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000bb29f-a137-4865-bd84-69d930f8faed None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:32.613663 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91585728-8eff-46bb-8a09-97fccad8ce99 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:32.647152 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d0969-457f-49c3-aee5-97466f8dbb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:32.647811 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d0969-457f-49c3-aee5-97466f8dbb45 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:32.648095 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d0969-457f-49c3-aee5-97466f8dbb45 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:32.648375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d0969-457f-49c3-aee5-97466f8dbb45 None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:32.649114 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d0969-457f-49c3-aee5-97466f8dbb45 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:32.649423 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d0969-457f-49c3-aee5-97466f8dbb45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:32.651688 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d0969-457f-49c3-aee5-97466f8dbb45 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:32.656528 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 493/972] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Jul 6 13:33:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:32.657947 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39452a8e-3e81-47b1-bae4-ef20ebee6c0c None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:32.658235 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39452a8e-3e81-47b1-bae4-ef20ebee6c0c None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:32.658509 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39452a8e-3e81-47b1-bae4-ef20ebee6c0c None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:32.659729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 494/973] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:32.664224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39452a8e-3e81-47b1-bae4-ef20ebee6c0c None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:32.682203 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:32.717619 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91585728-8eff-46bb-8a09-97fccad8ce99 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:32.751999 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a83e4f5-36e0-4d38-90a0-494b35b417a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:32.752712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83e4f5-36e0-4d38-90a0-494b35b417a3 None admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:32.753022 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83e4f5-36e0-4d38-90a0-494b35b417a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:32.753305 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83e4f5-36e0-4d38-90a0-494b35b417a3 None admin] PATH_INFO: `/v3/services` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:32.754023 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a83e4f5-36e0-4d38-90a0-494b35b417a3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:32.755445 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a83e4f5-36e0-4d38-90a0-494b35b417a3 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:32.769117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 480/974] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Jul 6 13:33:32 2022] POST /identity/v3/services => generated 242 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:33:32.770488 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d89a1-0d42-4b72-a882-2b1ab5e342a5 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:32.770779 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d89a1-0d42-4b72-a882-2b1ab5e342a5 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:32.771053 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d89a1-0d42-4b72-a882-2b1ab5e342a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:32.785374 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39452a8e-3e81-47b1-bae4-ef20ebee6c0c None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:32.791427 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-492d89a1-0d42-4b72-a882-2b1ab5e342a5 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:32.792650 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-492d89a1-0d42-4b72-a882-2b1ab5e342a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:32.826380 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba497a-dd38-4a0f-a73b-cd980b5dd0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:32.827066 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba497a-dd38-4a0f-a73b-cd980b5dd0ea None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:32.827349 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba497a-dd38-4a0f-a73b-cd980b5dd0ea None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:32.827624 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba497a-dd38-4a0f-a73b-cd980b5dd0ea None admin] PATH_INFO: `/v3/endpoints` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:32.828451 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ba497a-dd38-4a0f-a73b-cd980b5dd0ea None admin] RBAC: Adding query filter params (interface=admin, service_id=17625ac4b8a5490680fdf1d9ff84bffd, region_id=RegionOne) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:32.828764 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ba497a-dd38-4a0f-a73b-cd980b5dd0ea None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:32.830196 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ba497a-dd38-4a0f-a73b-cd980b5dd0ea None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:32.836274 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:32.836482 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 495/975] 10.222.0.20 () {56 vars in 1249 bytes} [Wed Jul 6 13:33:32 2022] GET /identity/v3/endpoints?service_id=17625ac4b8a5490680fdf1d9ff84bffd&interface=admin®ion_id=RegionOne => generated 193 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:32.842955 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba497a-dd38-4a0f-a73b-cd980b5dd0ea None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:32.879946 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 481/976] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:32 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:32.904480 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d89a1-0d42-4b72-a882-2b1ab5e342a5 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:32.935000 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:32.965682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba497a-dd38-4a0f-a73b-cd980b5dd0ea None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:33.003283 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ba552-51a1-4806-85c7-c0b389cc2fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:33.003946 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ba552-51a1-4806-85c7-c0b389cc2fec None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:33.004231 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ba552-51a1-4806-85c7-c0b389cc2fec None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:33.004499 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ba552-51a1-4806-85c7-c0b389cc2fec None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:33.005216 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ba552-51a1-4806-85c7-c0b389cc2fec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:33.006575 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ba552-51a1-4806-85c7-c0b389cc2fec None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:33.021036 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 496/977] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Jul 6 13:33:32 2022] GET /identity/v3/services => generated 1727 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:33:33.021270 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d89a1-0d42-4b72-a882-2b1ab5e342a5 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:33.068270 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6dcad2-6c6b-485a-aa78-71b7877bd196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:33.069142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6dcad2-6c6b-485a-aa78-71b7877bd196 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:33.069428 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6dcad2-6c6b-485a-aa78-71b7877bd196 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:33.069703 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6dcad2-6c6b-485a-aa78-71b7877bd196 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:33.082922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6dcad2-6c6b-485a-aa78-71b7877bd196 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:33:33.092269 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-5b6dcad2-6c6b-485a-aa78-71b7877bd196 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 13:33:33.092269 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:33.092269 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:33.092269 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:33.092269 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:33.092269 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:33.092269 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:33.092269 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:33.092269 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:33.092269 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:33.092269 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:33.092269 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:33.092269 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:33:33.092269 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:33:33.092269 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:33:33.092269 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:33:33.093385 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:33:33.093385 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:33:33.093385 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:33:33.093385 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:33:33.093385 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:33.093385 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:33.093385 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:33.093385 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:33.093385 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:33.093385 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:33.093385 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:33.093385 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:33.093385 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:33.093385 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:33.093385 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:33.093385 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:33.094353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:33.094353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:33.094353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:33.094353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:33.094353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:33:33.094353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:33:33.094353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:33:33.094353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:33:33.094353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:33:33.094353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:33:33.094353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 13:33:33.094353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:33.094353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 482/978] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Jul 6 13:33:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:33:33.099831 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ba552-51a1-4806-85c7-c0b389cc2fec None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:33.209959 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ba552-51a1-4806-85c7-c0b389cc2fec None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:33.246510 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f217307-3a43-4f1f-8db1-c0ac31b03fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:33.247479 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f217307-3a43-4f1f-8db1-c0ac31b03fef None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:33.247974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f217307-3a43-4f1f-8db1-c0ac31b03fef None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:33.248422 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f217307-3a43-4f1f-8db1-c0ac31b03fef None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:33.249376 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f217307-3a43-4f1f-8db1-c0ac31b03fef None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:33.249854 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f217307-3a43-4f1f-8db1-c0ac31b03fef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:33.251369 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f217307-3a43-4f1f-8db1-c0ac31b03fef None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:33.256772 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:33.263827 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 497/979] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Jul 6 13:33:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:33.267336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6dcad2-6c6b-485a-aa78-71b7877bd196 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:33.307874 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:33.385442 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6dcad2-6c6b-485a-aa78-71b7877bd196 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:33.422024 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eabcb4-f423-40ca-ae6a-1ad7826eab73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:33.422822 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eabcb4-f423-40ca-ae6a-1ad7826eab73 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:33.423104 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eabcb4-f423-40ca-ae6a-1ad7826eab73 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:33.423378 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eabcb4-f423-40ca-ae6a-1ad7826eab73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:33.434862 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eabcb4-f423-40ca-ae6a-1ad7826eab73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:33.436314 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eabcb4-f423-40ca-ae6a-1ad7826eab73 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:33.454491 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-87eabcb4-f423-40ca-ae6a-1ad7826eab73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:33.454491 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:33.454491 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:33.454491 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:33.454491 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:33.454491 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:33.454491 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:33.454491 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:33.454491 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:33.454491 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:33.454491 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:33.454491 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:33.454491 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:33.454491 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:33.454491 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:33.455533 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:33.455533 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:33.455533 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:33.455533 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:33.455533 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:33.455533 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:33.455533 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:33.455533 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:33.455533 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:33.455533 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:33.455533 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:33.455533 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:33.455533 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:33.455533 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:33.455533 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:33.455533 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:33.455533 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:33.461086 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:33.461086 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:33.461086 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:33.461086 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:33.461086 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 483/980] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:33.464038 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f217307-3a43-4f1f-8db1-c0ac31b03fef None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:33.566150 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f217307-3a43-4f1f-8db1-c0ac31b03fef None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:33.599728 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d8236f-f00a-4b82-a29b-ea48d54299a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:33.600391 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d8236f-f00a-4b82-a29b-ea48d54299a3 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:33.600675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d8236f-f00a-4b82-a29b-ea48d54299a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:33.600956 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d8236f-f00a-4b82-a29b-ea48d54299a3 None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:33.601708 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d8236f-f00a-4b82-a29b-ea48d54299a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:33.602018 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d8236f-f00a-4b82-a29b-ea48d54299a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:33.603499 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d8236f-f00a-4b82-a29b-ea48d54299a3 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:33.619838 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 498/981] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:33.621416 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eabcb4-f423-40ca-ae6a-1ad7826eab73 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:33.708352 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:33.726340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eabcb4-f423-40ca-ae6a-1ad7826eab73 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:33.759810 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f84be-ef52-489c-8019-3b507c4a6f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:33.760619 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f84be-ef52-489c-8019-3b507c4a6f95 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:33.760900 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f84be-ef52-489c-8019-3b507c4a6f95 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:33.761197 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f84be-ef52-489c-8019-3b507c4a6f95 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:33.769133 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f84be-ef52-489c-8019-3b507c4a6f95 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:33.770568 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f84be-ef52-489c-8019-3b507c4a6f95 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:33.786148 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-9e7f84be-ef52-489c-8019-3b507c4a6f95 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 13:33:33.786148 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:33.786148 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:33.786148 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:33.786148 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:33.786148 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:33.786148 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:33.786148 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:33.786148 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:33.786148 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:33.786148 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:33.786148 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:33.786148 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:33:33.786148 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:33:33.786148 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:33:33.786148 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:33:33.787282 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:33.787282 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:33.787282 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:33:33.787282 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:33.787282 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:33:33.787282 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:33.787282 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:33.787282 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:33.787282 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:33.787282 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:33.787282 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:33.787282 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:33.787282 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:33.787282 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:33.787282 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:33.787282 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:33.787282 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:33.787282 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:33.788272 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:33.788272 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:33.788272 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:33:33.788272 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:33:33.788272 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:33:33.788272 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:33:33.788272 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:33:33.788272 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:33:33.788272 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 13:33:33.788272 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:33.792295 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 484/982] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Jul 6 13:33:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:33.793704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d8236f-f00a-4b82-a29b-ea48d54299a3 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:33.863764 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:33.906542 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d8236f-f00a-4b82-a29b-ea48d54299a3 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:33.947626 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baea9fa-5d26-416a-b60f-d2f5454faf5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:33.948345 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baea9fa-5d26-416a-b60f-d2f5454faf5a None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:33.949023 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baea9fa-5d26-416a-b60f-d2f5454faf5a None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:33.949425 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baea9fa-5d26-416a-b60f-d2f5454faf5a None admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:33.950712 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baea9fa-5d26-416a-b60f-d2f5454faf5a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:33.951261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baea9fa-5d26-416a-b60f-d2f5454faf5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:33.953330 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baea9fa-5d26-416a-b60f-d2f5454faf5a None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:33.977396 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 499/983] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Jul 6 13:33:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:33.979767 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f84be-ef52-489c-8019-3b507c4a6f95 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:34.095746 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f84be-ef52-489c-8019-3b507c4a6f95 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:34.132159 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac595bf2-2c86-44b8-a887-c545417e6fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:34.132866 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac595bf2-2c86-44b8-a887-c545417e6fe3 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:34.133183 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac595bf2-2c86-44b8-a887-c545417e6fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:34.133457 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac595bf2-2c86-44b8-a887-c545417e6fe3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:34.151361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:34.211795 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac595bf2-2c86-44b8-a887-c545417e6fe3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:34.213475 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac595bf2-2c86-44b8-a887-c545417e6fe3 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:34.347258 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 485/984] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:33:33 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 13:33:34.348705 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baea9fa-5d26-416a-b60f-d2f5454faf5a None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:34.468704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baea9fa-5d26-416a-b60f-d2f5454faf5a None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:34.491119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:34.511448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae3b6c-fcc3-4a1f-a31d-6fadf9aae152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:34.512526 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae3b6c-fcc3-4a1f-a31d-6fadf9aae152 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:34.512955 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae3b6c-fcc3-4a1f-a31d-6fadf9aae152 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:34.513418 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae3b6c-fcc3-4a1f-a31d-6fadf9aae152 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:34.529649 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae3b6c-fcc3-4a1f-a31d-6fadf9aae152 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:34.531451 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae3b6c-fcc3-4a1f-a31d-6fadf9aae152 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:34.550914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-25ae3b6c-fcc3-4a1f-a31d-6fadf9aae152 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:34.550914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:34.550914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:34.550914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:34.550914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:34.550914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:34.550914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:34.550914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:34.550914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:34.550914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:34.550914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:34.550914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:34.550914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:34.550914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:34.550914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:34.551902 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:34.551902 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:34.551902 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:34.551902 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:34.551902 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:34.551902 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:34.551902 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:34.551902 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:34.551902 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:34.551902 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:34.551902 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:34.551902 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:34.551902 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:34.551902 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:34.551902 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:34.551902 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:34.551902 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:34.552807 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:34.552807 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:34.552807 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:34.552807 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:34.559286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 500/985] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:34.560784 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac595bf2-2c86-44b8-a887-c545417e6fe3 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:34.614566 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:34.678630 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac595bf2-2c86-44b8-a887-c545417e6fe3 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:34.731865 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e74628-61b3-4414-a146-72b0088002b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:34.732538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e74628-61b3-4414-a146-72b0088002b8 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:34.732827 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e74628-61b3-4414-a146-72b0088002b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:34.733126 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e74628-61b3-4414-a146-72b0088002b8 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:34.733858 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e74628-61b3-4414-a146-72b0088002b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:34.734172 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e74628-61b3-4414-a146-72b0088002b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:34.735538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e74628-61b3-4414-a146-72b0088002b8 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:34.751925 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 486/986] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:34.753469 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae3b6c-fcc3-4a1f-a31d-6fadf9aae152 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:34.855436 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae3b6c-fcc3-4a1f-a31d-6fadf9aae152 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:34.888994 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74c040-a19c-4c89-b495-1218d09c6a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:34.889814 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74c040-a19c-4c89-b495-1218d09c6a0e None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:34.890094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74c040-a19c-4c89-b495-1218d09c6a0e None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:34.890366 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74c040-a19c-4c89-b495-1218d09c6a0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:34.897289 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c74c040-a19c-4c89-b495-1218d09c6a0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:34.898727 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c74c040-a19c-4c89-b495-1218d09c6a0e None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:34.913726 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-8c74c040-a19c-4c89-b495-1218d09c6a0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:34.913726 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:34.913726 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:34.913726 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:34.913726 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:34.913726 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:34.913726 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:34.913726 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:34.913726 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:34.913726 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:34.913726 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:34.913726 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:34.913726 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:33:34.913726 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:33:34.913726 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:33:34.914634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:33:34.914634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:34.914634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:34.914634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:34.914634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:34.914634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:34.914634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:34.914634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:34.914634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:34.914634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:34.914634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:34.914634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:34.914634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:34.914634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:34.914634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:34.914634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:34.914634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:34.916036 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:33:34.916036 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:33:34.916036 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:33:34.916036 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:33:34.916036 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:33:34.916036 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:33:34.916036 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:34.916036 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:34.916036 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 501/987] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:33:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:33:34.924376 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e74628-61b3-4414-a146-72b0088002b8 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:34.934057 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479bb5b-4a53-42eb-8d7d-9e5e66ef811d None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:34.934365 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479bb5b-4a53-42eb-8d7d-9e5e66ef811d None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:34.934617 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479bb5b-4a53-42eb-8d7d-9e5e66ef811d None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:34.936171 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 502/988] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:34.941759 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19968d2-0d4f-471f-b328-110d642c5eea None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:34.942047 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19968d2-0d4f-471f-b328-110d642c5eea None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:34.942314 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19968d2-0d4f-471f-b328-110d642c5eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:34.959374 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-d19968d2-0d4f-471f-b328-110d642c5eea None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:34.960540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-d19968d2-0d4f-471f-b328-110d642c5eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:34.986154 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:35.030661 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e74628-61b3-4414-a146-72b0088002b8 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:35.031937 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 503/989] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:34 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 13:33:35.033335 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:35.054247 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19968d2-0d4f-471f-b328-110d642c5eea None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:35.068614 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e5e73-e9b1-4292-bb3d-f17b8ea483e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:35.069318 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e5e73-e9b1-4292-bb3d-f17b8ea483e1 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:35.069601 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e5e73-e9b1-4292-bb3d-f17b8ea483e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:35.069872 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e5e73-e9b1-4292-bb3d-f17b8ea483e1 None admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:35.070660 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e5e73-e9b1-4292-bb3d-f17b8ea483e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:35.070966 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e5e73-e9b1-4292-bb3d-f17b8ea483e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:35.073151 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e5e73-e9b1-4292-bb3d-f17b8ea483e1 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:35.090493 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 487/990] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:33:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:35.091554 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e5e73-e9b1-4292-bb3d-f17b8ea483e1 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:35.167915 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19968d2-0d4f-471f-b328-110d642c5eea None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:35.200790 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e5e73-e9b1-4292-bb3d-f17b8ea483e1 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:35.204722 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c1b08-8bbd-4c66-b309-5535827fd1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:35.205548 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c1b08-8bbd-4c66-b309-5535827fd1af None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:35.205832 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c1b08-8bbd-4c66-b309-5535827fd1af None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:35.206107 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c1b08-8bbd-4c66-b309-5535827fd1af None admin] PATH_INFO: `/v3/services/compute` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:35.206761 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c1b08-8bbd-4c66-b309-5535827fd1af None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:35.208526 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c1b08-8bbd-4c66-b309-5535827fd1af None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:35.221535 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-9e3c1b08-8bbd-4c66-b309-5535827fd1af None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 06 13:33:35.221535 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:35.221535 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 13:33:35.221535 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 13:33:35.221535 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 13:33:35.221535 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 13:33:35.221535 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 13:33:35.221535 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:35.221535 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 06 13:33:35.221535 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:35.221535 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 13:33:35.221535 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:35.221535 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:35.221535 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:35.221535 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:35.221535 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:35.221535 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:35.222664 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:35.222664 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:35.222664 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:35.222664 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:35.222664 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:35.222664 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:35.222664 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 13:33:35.222664 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 13:33:35.222664 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 13:33:35.222664 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 13:33:35.222664 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:35.222664 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:35.222664 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:35.222664 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:35.222664 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:35.222664 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:35.223516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:35.223516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:35.223516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:35.223516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:35.223516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:35.223516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:35.223516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:35.223516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:35.223516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:35.223516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:35.223516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 13:33:35.223516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:35.223516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 06 13:33:35.223516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:35.227556 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 504/991] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:33:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:33:35.228878 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89fb24-ff74-4ae4-b3fa-b01dd5458a07 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:35.229192 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89fb24-ff74-4ae4-b3fa-b01dd5458a07 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:35.229473 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89fb24-ff74-4ae4-b3fa-b01dd5458a07 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:35.230747 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 505/992] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:35.234614 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89fb24-ff74-4ae4-b3fa-b01dd5458a07 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:35.234971 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909ec1c-d3ff-402e-a176-bf87ab4502dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:35.235636 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909ec1c-d3ff-402e-a176-bf87ab4502dd None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:35.235917 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909ec1c-d3ff-402e-a176-bf87ab4502dd None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:35.236191 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909ec1c-d3ff-402e-a176-bf87ab4502dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:35.237039 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8909ec1c-d3ff-402e-a176-bf87ab4502dd None admin] RBAC: Adding query filter params (role.id=8d259fab1ed64ced91f18487e90ee097, scope.project.id=bfe6c3d8da2743be9609184db2506303, user.id=c4535134181e489fb532f617264d5855) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:35.237355 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8909ec1c-d3ff-402e-a176-bf87ab4502dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:35.238770 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8909ec1c-d3ff-402e-a176-bf87ab4502dd None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:35.245308 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 488/993] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:33:35 2022] GET /identity/v3/role_assignments?user.id=c4535134181e489fb532f617264d5855&scope.project.id=bfe6c3d8da2743be9609184db2506303&role.id=8d259fab1ed64ced91f18487e90ee097 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:35.246070 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8229c-64e7-4ed7-8024-100b031f138b None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:35.246360 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8229c-64e7-4ed7-8024-100b031f138b None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:35.246636 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8229c-64e7-4ed7-8024-100b031f138b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:35.260175 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:35.264949 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-6fd8229c-64e7-4ed7-8024-100b031f138b None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:35.266207 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-6fd8229c-64e7-4ed7-8024-100b031f138b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:35.292721 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:35.346114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 489/994] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:35.357429 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89fb24-ff74-4ae4-b3fa-b01dd5458a07 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:35.369952 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8229c-64e7-4ed7-8024-100b031f138b None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:35.394562 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f8c3c-8dfc-4308-a56d-6cc16dc27d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:35.395358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f8c3c-8dfc-4308-a56d-6cc16dc27d7a None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:35.395698 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f8c3c-8dfc-4308-a56d-6cc16dc27d7a None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:35.396101 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f8c3c-8dfc-4308-a56d-6cc16dc27d7a None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:35.396811 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f8c3c-8dfc-4308-a56d-6cc16dc27d7a None admin] RBAC: Adding query filter params (name=compute) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:35.397152 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f8c3c-8dfc-4308-a56d-6cc16dc27d7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:35.398629 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f8c3c-8dfc-4308-a56d-6cc16dc27d7a None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:35.409986 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 506/995] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Jul 6 13:33:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:35.411020 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f8c3c-8dfc-4308-a56d-6cc16dc27d7a None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:35.482443 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8229c-64e7-4ed7-8024-100b031f138b None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:35.516298 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:35.521390 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2daaf0-3fef-431d-9309-979e35bd5a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:35.522202 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2daaf0-3fef-431d-9309-979e35bd5a16 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:35.522501 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2daaf0-3fef-431d-9309-979e35bd5a16 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:35.522766 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2daaf0-3fef-431d-9309-979e35bd5a16 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:35.523433 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2daaf0-3fef-431d-9309-979e35bd5a16 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:35.524840 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2daaf0-3fef-431d-9309-979e35bd5a16 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:35.529643 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f8c3c-8dfc-4308-a56d-6cc16dc27d7a None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:35.537060 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-ee2daaf0-3fef-431d-9309-979e35bd5a16 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 13:33:35.537060 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:35.537060 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 13:33:35.537060 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 13:33:35.537060 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 13:33:35.537060 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 13:33:35.537060 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 13:33:35.537060 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:35.537060 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 13:33:35.537060 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:35.537060 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 13:33:35.537060 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:35.537060 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:35.537060 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:35.537060 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:35.537060 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:35.537060 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:35.538257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:35.538257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:35.538257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:35.538257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:35.538257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:35.538257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:35.538257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 13:33:35.538257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 13:33:35.538257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 13:33:35.538257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 13:33:35.538257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:35.538257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:35.538257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:35.538257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:35.538257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:35.538257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:35.539258 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:35.539258 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:35.539258 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:35.539258 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:35.539258 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:35.539258 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:35.539258 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:35.539258 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:35.539258 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:35.539258 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:35.539258 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 13:33:35.539258 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:35.539258 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 13:33:35.539258 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:35.539258 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 490/996] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Jul 6 13:33:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:33:35.544578 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2daaf0-3fef-431d-9309-979e35bd5a16 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:35.570924 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:35.576681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a46a5-e08c-4bbb-a3ff-cea4d9e62f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:35.577383 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a46a5-e08c-4bbb-a3ff-cea4d9e62f71 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:35.577661 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a46a5-e08c-4bbb-a3ff-cea4d9e62f71 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:35.577932 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a46a5-e08c-4bbb-a3ff-cea4d9e62f71 None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:35.578757 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a46a5-e08c-4bbb-a3ff-cea4d9e62f71 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:35.579178 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a46a5-e08c-4bbb-a3ff-cea4d9e62f71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:35.580540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a46a5-e08c-4bbb-a3ff-cea4d9e62f71 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:35.590886 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 507/997] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Jul 6 13:33:35 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:35.591926 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a46a5-e08c-4bbb-a3ff-cea4d9e62f71 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:35.661077 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2daaf0-3fef-431d-9309-979e35bd5a16 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:35.692916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a46a5-e08c-4bbb-a3ff-cea4d9e62f71 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:35.700144 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b41dc0e-00e1-4891-9333-b67455327c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:35.700818 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41dc0e-00e1-4891-9333-b67455327c20 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:35.701128 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41dc0e-00e1-4891-9333-b67455327c20 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:35.701403 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41dc0e-00e1-4891-9333-b67455327c20 None admin] PATH_INFO: `/v3/services` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:35.702107 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b41dc0e-00e1-4891-9333-b67455327c20 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:35.702413 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b41dc0e-00e1-4891-9333-b67455327c20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:35.703771 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b41dc0e-00e1-4891-9333-b67455327c20 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:35.713750 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 491/998] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Jul 6 13:33:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:35.714795 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b41dc0e-00e1-4891-9333-b67455327c20 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:35.731063 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef702b22-afbe-4517-8571-df2b4e79dfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:35.731724 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef702b22-afbe-4517-8571-df2b4e79dfe9 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:35.732019 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef702b22-afbe-4517-8571-df2b4e79dfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:35.732279 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef702b22-afbe-4517-8571-df2b4e79dfe9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:35.733152 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef702b22-afbe-4517-8571-df2b4e79dfe9 None admin] RBAC: Adding query filter params (interface=public, service_id=b98e5883075446bd98a2f1fc59b92f16, region_id=RegionOne) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:35.733409 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef702b22-afbe-4517-8571-df2b4e79dfe9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:35.734995 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef702b22-afbe-4517-8571-df2b4e79dfe9 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:35.744815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 508/999] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Jul 6 13:33:35 2022] GET /identity/v3/endpoints?service_id=b98e5883075446bd98a2f1fc59b92f16&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:35.745995 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef702b22-afbe-4517-8571-df2b4e79dfe9 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:35.765765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:35.802401 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:35.817733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b41dc0e-00e1-4891-9333-b67455327c20 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:35.852942 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537f2a67-9c11-4168-ad5b-de5c81591b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:35.853642 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f2a67-9c11-4168-ad5b-de5c81591b35 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:35.853932 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f2a67-9c11-4168-ad5b-de5c81591b35 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:35.854209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f2a67-9c11-4168-ad5b-de5c81591b35 None admin] PATH_INFO: `/v3/services` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:35.854916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537f2a67-9c11-4168-ad5b-de5c81591b35 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:35.855222 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537f2a67-9c11-4168-ad5b-de5c81591b35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:35.856591 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537f2a67-9c11-4168-ad5b-de5c81591b35 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:35.857375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef702b22-afbe-4517-8571-df2b4e79dfe9 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:35.861355 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 492/1000] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Jul 6 13:33:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:35.866820 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537f2a67-9c11-4168-ad5b-de5c81591b35 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:35.894429 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9543f-c408-4e4c-9aa6-5ea35c045467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:35.895099 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9543f-c408-4e4c-9aa6-5ea35c045467 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:35.895388 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9543f-c408-4e4c-9aa6-5ea35c045467 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:35.895671 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9543f-c408-4e4c-9aa6-5ea35c045467 None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:35.896355 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc9543f-c408-4e4c-9aa6-5ea35c045467 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:35.897759 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc9543f-c408-4e4c-9aa6-5ea35c045467 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:35.904344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 509/1001] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Jul 6 13:33:35 2022] GET /identity/v3/services => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:33:35.981155 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537f2a67-9c11-4168-ad5b-de5c81591b35 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:36.017252 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b80f61-f906-45f5-8fc1-3ed757fef644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:36.017937 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b80f61-f906-45f5-8fc1-3ed757fef644 None admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:36.018221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b80f61-f906-45f5-8fc1-3ed757fef644 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:36.018495 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b80f61-f906-45f5-8fc1-3ed757fef644 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:36.019247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b80f61-f906-45f5-8fc1-3ed757fef644 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:36.020626 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b80f61-f906-45f5-8fc1-3ed757fef644 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:36.038336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 493/1002] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Jul 6 13:33:35 2022] POST /identity/v3/endpoints => generated 325 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:33:37.320046 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabac334-4120-4f14-b3d4-d851ebbb5cb5 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:37.320538 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabac334-4120-4f14-b3d4-d851ebbb5cb5 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:37.320631 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabac334-4120-4f14-b3d4-d851ebbb5cb5 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:37.322785 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 510/1003] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:37.327785 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca35389-efcd-4e76-ad37-17c2c93bb62a None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:37.328164 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca35389-efcd-4e76-ad37-17c2c93bb62a None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:37.328340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca35389-efcd-4e76-ad37-17c2c93bb62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:37.345753 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-aca35389-efcd-4e76-ad37-17c2c93bb62a None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:37.346929 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-aca35389-efcd-4e76-ad37-17c2c93bb62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:37.349721 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:37.423184 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 494/1004] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:37 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:37.445343 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabac334-4120-4f14-b3d4-d851ebbb5cb5 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:37.530272 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:37.551150 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabac334-4120-4f14-b3d4-d851ebbb5cb5 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:37.585154 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632d1e7-dbb1-4282-896f-2140c69301f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:37.585942 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632d1e7-dbb1-4282-896f-2140c69301f3 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:37.586224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632d1e7-dbb1-4282-896f-2140c69301f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:37.586503 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632d1e7-dbb1-4282-896f-2140c69301f3 None admin] PATH_INFO: `/v3/services/image` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:37.587208 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5632d1e7-dbb1-4282-896f-2140c69301f3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:37.588677 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5632d1e7-dbb1-4282-896f-2140c69301f3 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:37.599992 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-5632d1e7-dbb1-4282-896f-2140c69301f3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 06 13:33:37.599992 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:37.599992 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 13:33:37.599992 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 13:33:37.599992 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 13:33:37.599992 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 13:33:37.599992 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 13:33:37.599992 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:37.599992 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 06 13:33:37.599992 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:37.599992 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 13:33:37.599992 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:37.599992 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:37.599992 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:37.599992 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:37.599992 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:37.599992 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:37.601025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:37.601025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:37.601025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:37.601025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:37.601025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:37.601025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:37.601025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 13:33:37.601025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 13:33:37.601025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 13:33:37.601025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 13:33:37.601025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:37.601025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:37.601025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:37.601025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:37.601025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:37.601025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:37.606771 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:37.606771 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:37.606771 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:37.606771 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:37.606771 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:37.606771 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:37.606771 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:37.606771 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:37.606771 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:37.606771 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:37.606771 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 13:33:37.606771 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:37.606771 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 06 13:33:37.606771 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:37.606771 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 511/1005] 10.222.0.20 () {56 vars in 1105 bytes} [Wed Jul 6 13:33:37 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:37.607668 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca35389-efcd-4e76-ad37-17c2c93bb62a None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:37.706102 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca35389-efcd-4e76-ad37-17c2c93bb62a None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:37.739471 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7944a841-ef9e-4a26-98b4-1b1093aef6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:37.740132 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944a841-ef9e-4a26-98b4-1b1093aef6f2 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:37.740415 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944a841-ef9e-4a26-98b4-1b1093aef6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:37.740689 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944a841-ef9e-4a26-98b4-1b1093aef6f2 None admin] PATH_INFO: `/v3/services` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:37.741418 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7944a841-ef9e-4a26-98b4-1b1093aef6f2 None admin] RBAC: Adding query filter params (name=image) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:37.741724 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7944a841-ef9e-4a26-98b4-1b1093aef6f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:37.743606 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7944a841-ef9e-4a26-98b4-1b1093aef6f2 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:37.745473 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:37.754673 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 495/1006] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:37.755761 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632d1e7-dbb1-4282-896f-2140c69301f3 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:37.855127 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632d1e7-dbb1-4282-896f-2140c69301f3 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:37.888587 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35644b4f-1176-4889-bd71-d41b8b200180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:37.889282 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35644b4f-1176-4889-bd71-d41b8b200180 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:37.889565 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35644b4f-1176-4889-bd71-d41b8b200180 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:37.889838 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35644b4f-1176-4889-bd71-d41b8b200180 None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:37.890537 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35644b4f-1176-4889-bd71-d41b8b200180 None admin] RBAC: Adding query filter params (type=image) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:37.890841 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35644b4f-1176-4889-bd71-d41b8b200180 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:37.892191 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35644b4f-1176-4889-bd71-d41b8b200180 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:37.905303 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 512/1007] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:37 2022] GET /identity/v3/services?type=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:38.025991 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089d635-494b-4352-bd6c-09a5ae33e3ba None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:38.026352 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089d635-494b-4352-bd6c-09a5ae33e3ba None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:38.026558 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089d635-494b-4352-bd6c-09a5ae33e3ba None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:38.028156 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 496/1008] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:38.033802 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f5a40-cc77-47e0-8bea-b2e6a94d16c2 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:38.034094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f5a40-cc77-47e0-8bea-b2e6a94d16c2 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:38.034378 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f5a40-cc77-47e0-8bea-b2e6a94d16c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:38.052588 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-5f8f5a40-cc77-47e0-8bea-b2e6a94d16c2 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:38.053795 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-5f8f5a40-cc77-47e0-8bea-b2e6a94d16c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:38.057143 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:38.079191 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8a65b-f6f3-4361-a106-155d5adb1c02 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:38.079525 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8a65b-f6f3-4361-a106-155d5adb1c02 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:38.079777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8a65b-f6f3-4361-a106-155d5adb1c02 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:38.081345 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 497/1009] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:33:38.087889 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f90363-8038-405a-89c8-c7e93c8eebd7 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:38.088175 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f90363-8038-405a-89c8-c7e93c8eebd7 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:38.088454 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f90363-8038-405a-89c8-c7e93c8eebd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:38.105248 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-02f90363-8038-405a-89c8-c7e93c8eebd7 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:38.106337 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-02f90363-8038-405a-89c8-c7e93c8eebd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:38.132056 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 513/1010] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:38 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:38.155515 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f5a40-cc77-47e0-8bea-b2e6a94d16c2 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:38.178478 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 498/1011] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:38 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 13:33:38.201964 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f90363-8038-405a-89c8-c7e93c8eebd7 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:38.263964 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f5a40-cc77-47e0-8bea-b2e6a94d16c2 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:38.302411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56299f8c-43ff-4a8c-8ab6-6502288d3758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:38.303221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56299f8c-43ff-4a8c-8ab6-6502288d3758 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:38.303508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56299f8c-43ff-4a8c-8ab6-6502288d3758 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:38.303788 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56299f8c-43ff-4a8c-8ab6-6502288d3758 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:38.304621 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56299f8c-43ff-4a8c-8ab6-6502288d3758 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:38.307327 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56299f8c-43ff-4a8c-8ab6-6502288d3758 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:38.319846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-56299f8c-43ff-4a8c-8ab6-6502288d3758 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 06 13:33:38.319846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:38.319846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 13:33:38.319846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 13:33:38.319846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 13:33:38.319846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 13:33:38.319846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 13:33:38.319846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:38.319846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 06 13:33:38.319846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:38.319846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 13:33:38.319846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:38.319846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:38.319846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:38.319846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:38.319846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:38.319846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:38.321947 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:38.321947 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:38.321947 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:38.321947 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:38.321947 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:38.321947 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:38.321947 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 13:33:38.321947 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 13:33:38.321947 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 13:33:38.321947 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 13:33:38.321947 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:38.321947 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:38.321947 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:38.321947 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:38.321947 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:38.321947 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:38.323002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:38.323002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:38.323002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:38.323002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:38.323002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:38.323002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:38.323002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:38.323002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:38.323002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:38.323002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:38.323002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 13:33:38.323002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:38.323002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 06 13:33:38.323002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:38.323002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 514/1012] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:33:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:33:38.327021 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56299f8c-43ff-4a8c-8ab6-6502288d3758 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:38.329271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f90363-8038-405a-89c8-c7e93c8eebd7 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:38.329887 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:38.361639 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:38.370301 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c6c57-6b92-480e-8e03-053687a09715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:38.371103 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c6c57-6b92-480e-8e03-053687a09715 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:38.371388 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c6c57-6b92-480e-8e03-053687a09715 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:38.371787 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c6c57-6b92-480e-8e03-053687a09715 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:38.372983 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c6c57-6b92-480e-8e03-053687a09715 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:38.374449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c6c57-6b92-480e-8e03-053687a09715 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:38.380649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 499/1013] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:38.381616 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c6c57-6b92-480e-8e03-053687a09715 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:38.436066 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56299f8c-43ff-4a8c-8ab6-6502288d3758 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:38.470606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bbba44-c5e0-42e6-805a-0110ae24c7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:38.471272 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbba44-c5e0-42e6-805a-0110ae24c7cb None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:38.471557 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbba44-c5e0-42e6-805a-0110ae24c7cb None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:38.471835 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbba44-c5e0-42e6-805a-0110ae24c7cb None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:38.472545 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bbba44-c5e0-42e6-805a-0110ae24c7cb None admin] RBAC: Adding query filter params (name=compute) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:38.472856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bbba44-c5e0-42e6-805a-0110ae24c7cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:38.474258 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bbba44-c5e0-42e6-805a-0110ae24c7cb None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:38.479745 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 515/1014] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Jul 6 13:33:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:38.485201 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bbba44-c5e0-42e6-805a-0110ae24c7cb None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:38.496613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c6c57-6b92-480e-8e03-053687a09715 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:38.532633 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d3873-ad26-4542-83ea-7bd09f02accf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:38.533317 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d3873-ad26-4542-83ea-7bd09f02accf None admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:38.533600 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d3873-ad26-4542-83ea-7bd09f02accf None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:38.533873 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d3873-ad26-4542-83ea-7bd09f02accf None admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:38.534674 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d3873-ad26-4542-83ea-7bd09f02accf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:38.536202 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d3873-ad26-4542-83ea-7bd09f02accf None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:38.550578 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:38.569100 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 500/1015] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Jul 6 13:33:38 2022] POST /identity/v3/projects => generated 311 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:33:38.591137 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bbba44-c5e0-42e6-805a-0110ae24c7cb None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:38.629588 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed04a109-40ef-402d-82b8-17571233c394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:38.630186 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04a109-40ef-402d-82b8-17571233c394 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:38.630470 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04a109-40ef-402d-82b8-17571233c394 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:38.630747 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04a109-40ef-402d-82b8-17571233c394 None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:38.631484 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed04a109-40ef-402d-82b8-17571233c394 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:38.631777 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed04a109-40ef-402d-82b8-17571233c394 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:38.633176 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed04a109-40ef-402d-82b8-17571233c394 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:38.644026 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 516/1016] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Jul 6 13:33:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:38.645650 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d3873-ad26-4542-83ea-7bd09f02accf None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:38.702614 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:38.759460 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d3873-ad26-4542-83ea-7bd09f02accf None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:38.794605 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0155dd47-8c66-45e9-b6aa-e4122708c694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:38.795632 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155dd47-8c66-45e9-b6aa-e4122708c694 None admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:38.796113 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155dd47-8c66-45e9-b6aa-e4122708c694 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:38.796608 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155dd47-8c66-45e9-b6aa-e4122708c694 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:38.797595 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0155dd47-8c66-45e9-b6aa-e4122708c694 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:38.799418 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0155dd47-8c66-45e9-b6aa-e4122708c694 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:38.817776 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 501/1017] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Jul 6 13:33:38 2022] POST /identity/v3/endpoints => generated 334 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:33:39.957936 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03560b55-98fb-43c0-a635-5d8c571ef6fa None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:39.958479 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03560b55-98fb-43c0-a635-5d8c571ef6fa None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:39.958479 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03560b55-98fb-43c0-a635-5d8c571ef6fa None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:39.960002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 517/1018] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:39.965639 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27851854-6e8f-406e-8678-f2c52f667489 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:39.966042 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27851854-6e8f-406e-8678-f2c52f667489 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:39.966229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27851854-6e8f-406e-8678-f2c52f667489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:39.982745 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-27851854-6e8f-406e-8678-f2c52f667489 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:39.983923 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-27851854-6e8f-406e-8678-f2c52f667489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:40.063310 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 502/1019] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:39 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:40.086372 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03560b55-98fb-43c0-a635-5d8c571ef6fa None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:40.189890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03560b55-98fb-43c0-a635-5d8c571ef6fa None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:40.218266 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:40.225347 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9306f65-0103-4ced-aab4-7b446e5578b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:40.226015 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9306f65-0103-4ced-aab4-7b446e5578b1 None admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:40.226303 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9306f65-0103-4ced-aab4-7b446e5578b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:40.226581 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9306f65-0103-4ced-aab4-7b446e5578b1 None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:40.227826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9306f65-0103-4ced-aab4-7b446e5578b1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:40.229220 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9306f65-0103-4ced-aab4-7b446e5578b1 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:40.249045 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 518/1020] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Jul 6 13:33:40 2022] POST /identity/v3/services => generated 242 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:33:40.642927 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e517b31-a66b-41a8-a92a-0d6bb617f6ea None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:40.643309 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e517b31-a66b-41a8-a92a-0d6bb617f6ea None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:40.643491 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e517b31-a66b-41a8-a92a-0d6bb617f6ea None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:40.644992 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 503/1021] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:40.650568 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ed205-fd57-4367-b875-ebe1866aec94 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:40.650865 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ed205-fd57-4367-b875-ebe1866aec94 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:40.651147 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ed205-fd57-4367-b875-ebe1866aec94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:40.668643 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-e48ed205-fd57-4367-b875-ebe1866aec94 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:40.669866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-e48ed205-fd57-4367-b875-ebe1866aec94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:40.746018 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 519/1022] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:40.770290 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e517b31-a66b-41a8-a92a-0d6bb617f6ea None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:40.807737 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:40.881798 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e517b31-a66b-41a8-a92a-0d6bb617f6ea None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:40.917100 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca20a6-ac06-4ab4-bfa2-07d495e4da09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:40.917804 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca20a6-ac06-4ab4-bfa2-07d495e4da09 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:40.918090 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca20a6-ac06-4ab4-bfa2-07d495e4da09 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:40.918367 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca20a6-ac06-4ab4-bfa2-07d495e4da09 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:40.919550 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca20a6-ac06-4ab4-bfa2-07d495e4da09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:40.921015 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca20a6-ac06-4ab4-bfa2-07d495e4da09 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:40.927451 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 504/1023] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:40.928659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ed205-fd57-4367-b875-ebe1866aec94 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:40.998182 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf74c8-af35-4fd1-8d42-b7dc27e63112 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:40.998727 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf74c8-af35-4fd1-8d42-b7dc27e63112 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:40.998727 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf74c8-af35-4fd1-8d42-b7dc27e63112 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:41.001130 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 505/1024] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:41.005288 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4092d-6347-4bbb-bb42-7271ccc8638e None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:41.005577 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4092d-6347-4bbb-bb42-7271ccc8638e None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:41.005856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4092d-6347-4bbb-bb42-7271ccc8638e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:41.023009 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-9fe4092d-6347-4bbb-bb42-7271ccc8638e None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:41.024295 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-9fe4092d-6347-4bbb-bb42-7271ccc8638e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:41.042554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ed205-fd57-4367-b875-ebe1866aec94 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:41.060425 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:41.079963 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd6481-50b0-4c77-92ec-e1ead3212065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:41.080634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd6481-50b0-4c77-92ec-e1ead3212065 None admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:41.080917 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd6481-50b0-4c77-92ec-e1ead3212065 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:41.081221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd6481-50b0-4c77-92ec-e1ead3212065 None admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:41.085032 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd6481-50b0-4c77-92ec-e1ead3212065 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:41.085032 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd6481-50b0-4c77-92ec-e1ead3212065 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:41.104663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 520/1025] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jul 6 13:33:40 2022] POST /identity/v3/users => generated 285 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:33:41.116398 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 506/1026] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 13:33:41.141904 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd6481-50b0-4c77-92ec-e1ead3212065 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:41.258039 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd6481-50b0-4c77-92ec-e1ead3212065 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:41.296353 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74151463-da7d-42a8-9487-a8aebfa5e07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:41.296861 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74151463-da7d-42a8-9487-a8aebfa5e07b None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:41.297219 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74151463-da7d-42a8-9487-a8aebfa5e07b None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:41.300539 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74151463-da7d-42a8-9487-a8aebfa5e07b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:41.310682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74151463-da7d-42a8-9487-a8aebfa5e07b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:33:41.318926 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-74151463-da7d-42a8-9487-a8aebfa5e07b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 13:33:41.318926 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:41.318926 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:41.318926 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:41.318926 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:41.318926 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:41.318926 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:41.318926 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:41.318926 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:41.318926 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:41.318926 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:41.318926 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:41.318926 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:33:41.318926 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:33:41.318926 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:33:41.318926 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:33:41.320043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:33:41.320043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:33:41.320043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:33:41.320043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:33:41.320043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:41.320043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:41.320043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:41.320043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:41.320043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:41.320043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:41.320043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:41.320043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:41.320043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:41.320043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:41.320043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:41.320043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:41.321140 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:41.321140 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:41.321140 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:41.321140 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:41.321140 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:33:41.321140 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:33:41.321140 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:33:41.321140 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:33:41.321140 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:33:41.321140 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:33:41.321140 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 13:33:41.321140 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:41.321140 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 521/1027] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Jul 6 13:33:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:41.326589 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4092d-6347-4bbb-bb42-7271ccc8638e None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:41.365250 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:41.441393 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4092d-6347-4bbb-bb42-7271ccc8638e None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:41.478221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3161dab6-4c74-456f-a9bf-2011d074b4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:41.478891 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3161dab6-4c74-456f-a9bf-2011d074b4c4 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:41.479177 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3161dab6-4c74-456f-a9bf-2011d074b4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:41.479455 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3161dab6-4c74-456f-a9bf-2011d074b4c4 None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:41.480208 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3161dab6-4c74-456f-a9bf-2011d074b4c4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:41.480514 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3161dab6-4c74-456f-a9bf-2011d074b4c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:41.481899 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3161dab6-4c74-456f-a9bf-2011d074b4c4 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:41.490253 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 507/1028] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Jul 6 13:33:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:41.496409 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74151463-da7d-42a8-9487-a8aebfa5e07b None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:41.528988 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:41.600832 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74151463-da7d-42a8-9487-a8aebfa5e07b None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:41.635326 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5aae2-95e6-44fd-bef8-3d892eb40a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:41.636120 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5aae2-95e6-44fd-bef8-3d892eb40a9c None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:41.636403 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5aae2-95e6-44fd-bef8-3d892eb40a9c None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:41.636677 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5aae2-95e6-44fd-bef8-3d892eb40a9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:41.647220 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b5aae2-95e6-44fd-bef8-3d892eb40a9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:41.648662 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b5aae2-95e6-44fd-bef8-3d892eb40a9c None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:41.667291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-55b5aae2-95e6-44fd-bef8-3d892eb40a9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:41.667291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:41.667291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:41.667291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:41.667291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:41.667291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:41.667291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:41.667291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:41.667291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:41.667291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:41.667291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:41.667291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:41.667291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:41.667291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:41.667291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:41.668211 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:41.668211 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:41.668211 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:41.668211 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:41.668211 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:41.668211 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:41.668211 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:41.668211 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:41.668211 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:41.668211 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:41.668211 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:41.668211 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:41.668211 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:41.668211 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:41.668211 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:41.668211 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:41.668211 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:41.669572 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:41.669572 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:41.669572 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:41.669572 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:41.669572 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 522/1029] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:41.674591 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3161dab6-4c74-456f-a9bf-2011d074b4c4 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:41.772633 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:41.783117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3161dab6-4c74-456f-a9bf-2011d074b4c4 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:41.816000 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33e5d6-85c4-4a13-ba4b-52e97d5e2c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:41.816662 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33e5d6-85c4-4a13-ba4b-52e97d5e2c24 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:41.816949 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33e5d6-85c4-4a13-ba4b-52e97d5e2c24 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:41.817254 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33e5d6-85c4-4a13-ba4b-52e97d5e2c24 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:41.817978 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b33e5d6-85c4-4a13-ba4b-52e97d5e2c24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:41.818303 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b33e5d6-85c4-4a13-ba4b-52e97d5e2c24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:41.819656 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b33e5d6-85c4-4a13-ba4b-52e97d5e2c24 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:41.836660 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 508/1030] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:41.838442 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5aae2-95e6-44fd-bef8-3d892eb40a9c None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:41.923856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:41.941049 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5aae2-95e6-44fd-bef8-3d892eb40a9c None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:41.975447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54c1cf-da07-44f6-8d5b-d4147cf8f4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:41.976234 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54c1cf-da07-44f6-8d5b-d4147cf8f4ee None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:41.976521 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54c1cf-da07-44f6-8d5b-d4147cf8f4ee None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:41.976793 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54c1cf-da07-44f6-8d5b-d4147cf8f4ee None admin] PATH_INFO: `/v3/users/nova` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:41.985207 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da54c1cf-da07-44f6-8d5b-d4147cf8f4ee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:41.986623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da54c1cf-da07-44f6-8d5b-d4147cf8f4ee None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:42.000860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-da54c1cf-da07-44f6-8d5b-d4147cf8f4ee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 06 13:33:42.000860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:42.000860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:42.000860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:42.000860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:42.000860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:42.000860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:42.000860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:42.000860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:42.000860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:42.000860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:42.000860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:42.000860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:33:42.000860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:33:42.000860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:33:42.000860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:33:42.001979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:42.001979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:42.001979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:33:42.001979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:42.001979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:33:42.001979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:42.001979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:42.001979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:42.001979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:42.001979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:42.001979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:42.001979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:42.001979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:42.001979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:42.001979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:42.001979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:42.001979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:42.001979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:42.008015 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:42.008015 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:42.008015 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:33:42.008015 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:33:42.008015 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:33:42.008015 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:33:42.008015 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:33:42.008015 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:33:42.008015 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 06 13:33:42.008015 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:42.008015 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 523/1031] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Jul 6 13:33:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:33:42.010082 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33e5d6-85c4-4a13-ba4b-52e97d5e2c24 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:42.113706 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33e5d6-85c4-4a13-ba4b-52e97d5e2c24 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:42.147302 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5467a6-b2bf-49f7-970b-904738f7941e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:42.147969 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5467a6-b2bf-49f7-970b-904738f7941e None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:42.148251 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5467a6-b2bf-49f7-970b-904738f7941e None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:42.148523 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5467a6-b2bf-49f7-970b-904738f7941e None admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:42.150463 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5467a6-b2bf-49f7-970b-904738f7941e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:42.150777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5467a6-b2bf-49f7-970b-904738f7941e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:42.152207 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5467a6-b2bf-49f7-970b-904738f7941e None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:42.178398 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 509/1032] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Jul 6 13:33:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:42.179679 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54c1cf-da07-44f6-8d5b-d4147cf8f4ee None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:42.284819 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54c1cf-da07-44f6-8d5b-d4147cf8f4ee None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:42.317311 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b385b5f-8534-458b-8c97-31e99f9ea0cc None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:42.317992 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b385b5f-8534-458b-8c97-31e99f9ea0cc None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:42.318273 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b385b5f-8534-458b-8c97-31e99f9ea0cc None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:42.320549 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 510/1033] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:42.327136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d5c47-8952-4dd1-96f8-f5e3d5a12d36 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:42.327488 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d5c47-8952-4dd1-96f8-f5e3d5a12d36 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:42.327681 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d5c47-8952-4dd1-96f8-f5e3d5a12d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:42.331000 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d0fe5-fbb2-4067-aa27-029a064a18a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:42.331730 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d0fe5-fbb2-4067-aa27-029a064a18a1 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:42.332012 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d0fe5-fbb2-4067-aa27-029a064a18a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:42.332297 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d0fe5-fbb2-4067-aa27-029a064a18a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:42.345441 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-3e3d5c47-8952-4dd1-96f8-f5e3d5a12d36 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:42.346805 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-3e3d5c47-8952-4dd1-96f8-f5e3d5a12d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:42.348837 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:42.350652 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:42.411906 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d0fe5-fbb2-4067-aa27-029a064a18a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:42.413291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d0fe5-fbb2-4067-aa27-029a064a18a1 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:42.421521 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 511/1034] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 13:33:42.444375 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3d5c47-8952-4dd1-96f8-f5e3d5a12d36 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:42.523013 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 524/1035] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:33:42 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 13:33:42.523944 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d0fe5-fbb2-4067-aa27-029a064a18a1 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:42.546508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3d5c47-8952-4dd1-96f8-f5e3d5a12d36 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:42.571631 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:42.586762 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e2424-6637-4ed6-9d37-2155dae21cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:42.587736 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e2424-6637-4ed6-9d37-2155dae21cfd None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:42.588133 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e2424-6637-4ed6-9d37-2155dae21cfd None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:42.588407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e2424-6637-4ed6-9d37-2155dae21cfd None admin] PATH_INFO: `/v3/services/image` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:42.589078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e2424-6637-4ed6-9d37-2155dae21cfd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:42.590435 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e2424-6637-4ed6-9d37-2155dae21cfd None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:42.602997 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-e90e2424-6637-4ed6-9d37-2155dae21cfd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 06 13:33:42.602997 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:42.602997 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 13:33:42.602997 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 13:33:42.602997 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 13:33:42.602997 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 13:33:42.602997 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 13:33:42.602997 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:42.602997 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 06 13:33:42.602997 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:42.602997 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 13:33:42.602997 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:42.602997 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:42.602997 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:42.602997 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:42.602997 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:42.602997 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:42.604041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:42.604041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:42.604041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:42.604041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:42.604041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:42.604041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:42.604041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 13:33:42.604041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 13:33:42.604041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 13:33:42.604041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 13:33:42.604041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:42.604041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:42.604041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:42.604041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:42.604041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:42.604041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:42.604879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:42.604879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:42.604879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:42.604879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:42.604879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:42.604879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:42.604879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:42.604879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:42.604879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:42.604879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:42.604879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 13:33:42.604879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:42.604879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 06 13:33:42.604879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:42.604879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 512/1036] 10.222.0.20 () {56 vars in 1105 bytes} [Wed Jul 6 13:33:42 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:33:42.610521 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e2424-6637-4ed6-9d37-2155dae21cfd None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:42.614031 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:42.630693 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d0fe5-fbb2-4067-aa27-029a064a18a1 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:42.666589 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648a720a-f125-4240-a360-51262680c930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:42.667302 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a720a-f125-4240-a360-51262680c930 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:42.667588 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a720a-f125-4240-a360-51262680c930 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:42.667867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a720a-f125-4240-a360-51262680c930 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:42.678084 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648a720a-f125-4240-a360-51262680c930 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:42.679518 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648a720a-f125-4240-a360-51262680c930 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:42.698508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-648a720a-f125-4240-a360-51262680c930 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:42.698508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:42.698508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:42.698508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:42.698508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:42.698508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:42.698508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:42.698508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:42.698508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:42.698508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:42.698508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:42.698508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:42.698508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:42.698508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:42.698508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:42.699461 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:42.699461 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:42.699461 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:42.699461 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:42.699461 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:42.699461 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:42.699461 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:42.699461 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:42.699461 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:42.699461 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:42.699461 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:42.699461 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:42.699461 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:42.699461 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:42.699461 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:42.699461 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:42.699461 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:42.701171 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:42.701171 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:42.701171 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:42.701171 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:42.701171 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 525/1037] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:42.705202 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648a720a-f125-4240-a360-51262680c930 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:42.719757 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e2424-6637-4ed6-9d37-2155dae21cfd None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:42.754050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143cdd54-a306-4064-b6b2-edad5ae8e3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:42.754706 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cdd54-a306-4064-b6b2-edad5ae8e3e2 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:42.754989 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cdd54-a306-4064-b6b2-edad5ae8e3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:42.755266 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cdd54-a306-4064-b6b2-edad5ae8e3e2 None admin] PATH_INFO: `/v3/services` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:42.755967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143cdd54-a306-4064-b6b2-edad5ae8e3e2 None admin] RBAC: Adding query filter params (name=image) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:42.756273 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143cdd54-a306-4064-b6b2-edad5ae8e3e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:42.757657 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143cdd54-a306-4064-b6b2-edad5ae8e3e2 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:42.767420 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 513/1038] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:42 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:42.769311 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143cdd54-a306-4064-b6b2-edad5ae8e3e2 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:42.816106 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:42.823788 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648a720a-f125-4240-a360-51262680c930 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:42.841090 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:42.864048 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa99150-dc3b-4d9f-8bff-372eea7456f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:42.864718 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa99150-dc3b-4d9f-8bff-372eea7456f2 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:42.865027 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa99150-dc3b-4d9f-8bff-372eea7456f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:42.865294 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa99150-dc3b-4d9f-8bff-372eea7456f2 None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:42.866018 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa99150-dc3b-4d9f-8bff-372eea7456f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:42.866323 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa99150-dc3b-4d9f-8bff-372eea7456f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:42.867938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa99150-dc3b-4d9f-8bff-372eea7456f2 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:42.876301 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143cdd54-a306-4064-b6b2-edad5ae8e3e2 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:42.884657 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 526/1039] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:42.885769 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa99150-dc3b-4d9f-8bff-372eea7456f2 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:42.914512 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c2740-c9cd-4a35-9c4a-3bc2cd9dbdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:42.915741 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c2740-c9cd-4a35-9c4a-3bc2cd9dbdb9 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:42.916017 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c2740-c9cd-4a35-9c4a-3bc2cd9dbdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:42.916297 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c2740-c9cd-4a35-9c4a-3bc2cd9dbdb9 None admin] PATH_INFO: `/v3/services` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:42.917077 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2c2740-c9cd-4a35-9c4a-3bc2cd9dbdb9 None admin] RBAC: Adding query filter params (type=image) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:42.917335 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2c2740-c9cd-4a35-9c4a-3bc2cd9dbdb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:42.918705 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2c2740-c9cd-4a35-9c4a-3bc2cd9dbdb9 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:42.925036 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 514/1040] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:42 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:42.930188 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c2740-c9cd-4a35-9c4a-3bc2cd9dbdb9 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:43.000715 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa99150-dc3b-4d9f-8bff-372eea7456f2 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:43.036497 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34178d85-cdd6-49f5-a4be-56a70dbfccf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:43.037345 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34178d85-cdd6-49f5-a4be-56a70dbfccf9 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:43.037633 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34178d85-cdd6-49f5-a4be-56a70dbfccf9 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:43.037915 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34178d85-cdd6-49f5-a4be-56a70dbfccf9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:43.040650 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c2740-c9cd-4a35-9c4a-3bc2cd9dbdb9 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:43.045131 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34178d85-cdd6-49f5-a4be-56a70dbfccf9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:43.046599 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34178d85-cdd6-49f5-a4be-56a70dbfccf9 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:43.062128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-34178d85-cdd6-49f5-a4be-56a70dbfccf9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:43.062128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:43.062128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:43.062128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:43.062128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:43.062128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:43.062128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:43.062128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:43.062128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:43.062128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:43.062128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:43.062128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:43.062128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:33:43.062128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:33:43.062128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:33:43.063064 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:33:43.063064 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:43.063064 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:43.063064 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:43.063064 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:43.063064 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:43.063064 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:43.063064 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:43.063064 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:43.063064 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:43.063064 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:43.063064 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:43.063064 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:43.063064 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:43.063064 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:43.063064 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:43.063064 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:43.063965 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:33:43.063965 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:33:43.063965 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:33:43.063965 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:33:43.063965 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:33:43.063965 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:33:43.063965 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:43.063965 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:43.063965 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 527/1041] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:33:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:43.069249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34178d85-cdd6-49f5-a4be-56a70dbfccf9 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:43.075208 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:43.080723 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b200a37-00db-4c69-b312-1f99443c7b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:43.081418 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b200a37-00db-4c69-b312-1f99443c7b84 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:43.081707 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b200a37-00db-4c69-b312-1f99443c7b84 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:43.081986 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b200a37-00db-4c69-b312-1f99443c7b84 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:43.082780 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b200a37-00db-4c69-b312-1f99443c7b84 None admin] RBAC: Adding query filter params (interface=public, service_id=126a95e5b8374f2bbc7c37661059806f, region_id=RegionOne) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:43.083100 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b200a37-00db-4c69-b312-1f99443c7b84 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:43.084458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b200a37-00db-4c69-b312-1f99443c7b84 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:43.088147 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:43.095686 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 515/1042] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Jul 6 13:33:42 2022] GET /identity/v3/endpoints?service_id=126a95e5b8374f2bbc7c37661059806f&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:43.096690 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b200a37-00db-4c69-b312-1f99443c7b84 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:43.188620 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34178d85-cdd6-49f5-a4be-56a70dbfccf9 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:43.208046 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b200a37-00db-4c69-b312-1f99443c7b84 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:43.228215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ccbec-7060-4398-a9eb-0756b021ddb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:43.228887 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ccbec-7060-4398-a9eb-0756b021ddb3 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:43.229203 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ccbec-7060-4398-a9eb-0756b021ddb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:43.229485 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ccbec-7060-4398-a9eb-0756b021ddb3 None admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:43.230467 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6ccbec-7060-4398-a9eb-0756b021ddb3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:43.230591 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6ccbec-7060-4398-a9eb-0756b021ddb3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:43.232007 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6ccbec-7060-4398-a9eb-0756b021ddb3 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:43.244371 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 528/1043] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:33:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:43.249815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ccbec-7060-4398-a9eb-0756b021ddb3 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:43.250703 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955d5ea-91e5-45a7-b9c0-e331196d1e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:43.251378 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955d5ea-91e5-45a7-b9c0-e331196d1e74 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:43.251662 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955d5ea-91e5-45a7-b9c0-e331196d1e74 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:43.251938 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955d5ea-91e5-45a7-b9c0-e331196d1e74 None admin] PATH_INFO: `/v3/services` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:43.252625 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955d5ea-91e5-45a7-b9c0-e331196d1e74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:43.254032 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955d5ea-91e5-45a7-b9c0-e331196d1e74 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:43.273040 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 516/1044] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Jul 6 13:33:43 2022] GET /identity/v3/services => generated 1957 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:33:43.286979 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39da92c-024c-4258-8f73-4e4e3c03d81b None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:43.287305 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39da92c-024c-4258-8f73-4e4e3c03d81b None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:43.287529 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39da92c-024c-4258-8f73-4e4e3c03d81b None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:43.297036 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 517/1045] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:33:43.297036 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f40d9-11fe-4863-9c60-2479f9bacc57 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:43.297036 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f40d9-11fe-4863-9c60-2479f9bacc57 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:43.297036 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f40d9-11fe-4863-9c60-2479f9bacc57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:43.316855 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-0c4f40d9-11fe-4863-9c60-2479f9bacc57 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:43.318233 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-0c4f40d9-11fe-4863-9c60-2479f9bacc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:43.339566 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:43.367312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ccbec-7060-4398-a9eb-0756b021ddb3 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:43.386501 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:43.407794 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be069f5-a52d-4908-bf30-a5441572f9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:43.408464 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be069f5-a52d-4908-bf30-a5441572f9d2 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:43.408747 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be069f5-a52d-4908-bf30-a5441572f9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:43.409049 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be069f5-a52d-4908-bf30-a5441572f9d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:43.409562 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 518/1046] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:43.409902 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be069f5-a52d-4908-bf30-a5441572f9d2 None admin] RBAC: Adding query filter params (role.id=8d259fab1ed64ced91f18487e90ee097, scope.project.id=bfe6c3d8da2743be9609184db2506303, user.id=6647eb1e8a1544ef9d3a9806f2d1fc51) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:43.410215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be069f5-a52d-4908-bf30-a5441572f9d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:43.411644 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be069f5-a52d-4908-bf30-a5441572f9d2 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:43.421067 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 529/1047] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:33:43 2022] GET /identity/v3/role_assignments?user.id=6647eb1e8a1544ef9d3a9806f2d1fc51&scope.project.id=bfe6c3d8da2743be9609184db2506303&role.id=8d259fab1ed64ced91f18487e90ee097 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:43.426902 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f40d9-11fe-4863-9c60-2479f9bacc57 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:43.560957 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f40d9-11fe-4863-9c60-2479f9bacc57 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:43.598358 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb4a72-8022-4aaa-96bb-3e35d42cb869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:43.599160 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb4a72-8022-4aaa-96bb-3e35d42cb869 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:43.599445 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb4a72-8022-4aaa-96bb-3e35d42cb869 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:43.599722 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb4a72-8022-4aaa-96bb-3e35d42cb869 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:43.610650 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eb4a72-8022-4aaa-96bb-3e35d42cb869 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:33:43.619458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-f8eb4a72-8022-4aaa-96bb-3e35d42cb869 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:33:43.619458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:43.619458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:43.619458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:43.619458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:43.619458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:43.619458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:43.619458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:43.619458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:43.619458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:43.619458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:43.619458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:43.619458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:33:43.619458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:33:43.619458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:33:43.619458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:33:43.620427 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:33:43.620427 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:33:43.620427 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:33:43.620427 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:33:43.620427 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:43.620427 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:43.620427 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:43.620427 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:43.620427 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:43.620427 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:43.620427 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:43.620427 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:43.620427 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:43.620427 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:43.620427 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:43.620427 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:43.621312 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:43.621312 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:43.621312 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:43.621312 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:43.621312 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:33:43.621312 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:33:43.621312 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:33:43.621312 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:33:43.621312 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:33:43.621312 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:33:43.621312 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:33:43.621312 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:43.625071 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 519/1048] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:33:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:43.626915 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be069f5-a52d-4908-bf30-a5441572f9d2 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:43.730422 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be069f5-a52d-4908-bf30-a5441572f9d2 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:43.766024 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f71af6-bd78-41e6-94c9-785ed8c217cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:43.766696 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f71af6-bd78-41e6-94c9-785ed8c217cd None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:43.766978 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f71af6-bd78-41e6-94c9-785ed8c217cd None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:43.767251 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f71af6-bd78-41e6-94c9-785ed8c217cd None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:43.767997 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f71af6-bd78-41e6-94c9-785ed8c217cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:43.768303 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f71af6-bd78-41e6-94c9-785ed8c217cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:43.769675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f71af6-bd78-41e6-94c9-785ed8c217cd None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:43.778884 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 530/1049] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:33:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:43.784343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb4a72-8022-4aaa-96bb-3e35d42cb869 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:43.842692 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:43.894741 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb4a72-8022-4aaa-96bb-3e35d42cb869 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:43.928953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33082423-be7f-46a0-8804-143f053ea6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:43.929971 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33082423-be7f-46a0-8804-143f053ea6b7 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:43.930261 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33082423-be7f-46a0-8804-143f053ea6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:43.930609 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33082423-be7f-46a0-8804-143f053ea6b7 None admin] PATH_INFO: `/v3/users/3e2a815b683e465e91713f548a64b13f` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:43.945235 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33082423-be7f-46a0-8804-143f053ea6b7 None admin] RBAC: Authorizing `identity:get_user(user_id=3e2a815b683e465e91713f548a64b13f)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:43.947200 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33082423-be7f-46a0-8804-143f053ea6b7 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:43.949098 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 520/1050] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:33:43 2022] GET /identity/v3/users/3e2a815b683e465e91713f548a64b13f => generated 285 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:43.954589 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f71af6-bd78-41e6-94c9-785ed8c217cd None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:43.957442 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:44.057318 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f71af6-bd78-41e6-94c9-785ed8c217cd None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:44.091070 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648bb68f-4e6c-4b39-9ac7-11e873a045ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:44.091723 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bb68f-4e6c-4b39-9ac7-11e873a045ad None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:44.092009 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bb68f-4e6c-4b39-9ac7-11e873a045ad None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:44.092289 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bb68f-4e6c-4b39-9ac7-11e873a045ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:44.160967 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648bb68f-4e6c-4b39-9ac7-11e873a045ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:44.162359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648bb68f-4e6c-4b39-9ac7-11e873a045ad None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:44.164193 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:44.267244 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 531/1051] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:33:43 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 13:33:44.268380 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33082423-be7f-46a0-8804-143f053ea6b7 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:44.376755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33082423-be7f-46a0-8804-143f053ea6b7 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:44.386745 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:44.412861 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f311d3e9-dbd3-46ec-9175-6a0d8e42a60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:44.413689 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311d3e9-dbd3-46ec-9175-6a0d8e42a60f None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:44.413977 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311d3e9-dbd3-46ec-9175-6a0d8e42a60f None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:44.414256 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311d3e9-dbd3-46ec-9175-6a0d8e42a60f None admin] PATH_INFO: `/v3/projects/9fa3428bf0d3459a8e7e86d666e373e6` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:44.416103 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f311d3e9-dbd3-46ec-9175-6a0d8e42a60f None admin] RBAC: Authorizing `identity:get_project(project_id=9fa3428bf0d3459a8e7e86d666e373e6)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:44.417632 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f311d3e9-dbd3-46ec-9175-6a0d8e42a60f None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:44.419689 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 521/1052] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:33:44 2022] GET /identity/v3/projects/9fa3428bf0d3459a8e7e86d666e373e6 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:44.426084 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648bb68f-4e6c-4b39-9ac7-11e873a045ad None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:44.534496 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648bb68f-4e6c-4b39-9ac7-11e873a045ad None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:44.536275 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:44.571975 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764a5ca-9c86-4375-9659-736a350845c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:44.572635 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764a5ca-9c86-4375-9659-736a350845c5 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:44.572921 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764a5ca-9c86-4375-9659-736a350845c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:44.573225 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764a5ca-9c86-4375-9659-736a350845c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:44.574065 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764a5ca-9c86-4375-9659-736a350845c5 None admin] RBAC: Adding query filter params (role.id=795daa2c78354fd48e4a741e1182cc98, scope.project.id=9fa3428bf0d3459a8e7e86d666e373e6, user.id=3e2a815b683e465e91713f548a64b13f) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:44.574377 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764a5ca-9c86-4375-9659-736a350845c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:44.575787 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764a5ca-9c86-4375-9659-736a350845c5 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:44.589036 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 532/1053] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:33:44 2022] GET /identity/v3/role_assignments?user.id=3e2a815b683e465e91713f548a64b13f&scope.project.id=9fa3428bf0d3459a8e7e86d666e373e6&role.id=795daa2c78354fd48e4a741e1182cc98 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:45.366647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3311df3-e9f4-414f-b98e-3e606ddcb171 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:45.367158 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3311df3-e9f4-414f-b98e-3e606ddcb171 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:45.367234 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3311df3-e9f4-414f-b98e-3e606ddcb171 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:45.368860 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 522/1054] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:45.373881 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9681b77-8172-4be5-89d7-a44e95f22a05 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:45.374324 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9681b77-8172-4be5-89d7-a44e95f22a05 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:45.374465 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9681b77-8172-4be5-89d7-a44e95f22a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:45.390782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-d9681b77-8172-4be5-89d7-a44e95f22a05 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:45.391940 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-d9681b77-8172-4be5-89d7-a44e95f22a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:45.457631 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce2ba2-ca42-48fb-9f27-8749885892e9 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:45.458015 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce2ba2-ca42-48fb-9f27-8749885892e9 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:45.458367 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce2ba2-ca42-48fb-9f27-8749885892e9 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:45.460107 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 523/1055] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:33:45.461102 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 533/1056] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:45 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:45.466207 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd019b9-e5e8-4d1a-96c0-b52d5a796e08 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:45.466496 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd019b9-e5e8-4d1a-96c0-b52d5a796e08 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:45.466772 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd019b9-e5e8-4d1a-96c0-b52d5a796e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:45.483751 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-3bd019b9-e5e8-4d1a-96c0-b52d5a796e08 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:45.485008 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-3bd019b9-e5e8-4d1a-96c0-b52d5a796e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:45.485821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9681b77-8172-4be5-89d7-a44e95f22a05 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:45.568158 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 524/1057] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:45 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 13:33:45.592233 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd019b9-e5e8-4d1a-96c0-b52d5a796e08 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:45.600510 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:45.607740 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9681b77-8172-4be5-89d7-a44e95f22a05 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:45.645104 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5dd5bd-4d51-4f30-b962-146fbf8029a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:45.645838 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5dd5bd-4d51-4f30-b962-146fbf8029a1 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:45.646116 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5dd5bd-4d51-4f30-b962-146fbf8029a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:45.646388 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5dd5bd-4d51-4f30-b962-146fbf8029a1 None admin] PATH_INFO: `/v3/services/image` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:45.647036 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5dd5bd-4d51-4f30-b962-146fbf8029a1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:45.648392 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5dd5bd-4d51-4f30-b962-146fbf8029a1 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:45.660672 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-8f5dd5bd-4d51-4f30-b962-146fbf8029a1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 06 13:33:45.660672 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:45.660672 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 13:33:45.660672 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 13:33:45.660672 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 13:33:45.660672 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 13:33:45.660672 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 13:33:45.660672 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:45.660672 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 06 13:33:45.660672 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:45.660672 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 13:33:45.660672 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:45.660672 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:45.660672 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:45.660672 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:45.660672 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:45.660672 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:45.661841 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:45.661841 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:45.661841 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:45.661841 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:45.661841 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:45.661841 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:45.661841 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 13:33:45.661841 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 13:33:45.661841 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 13:33:45.661841 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 13:33:45.661841 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:45.661841 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:45.661841 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:45.661841 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:45.661841 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:45.661841 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:45.662706 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:45.662706 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:45.662706 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:45.662706 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:45.662706 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:45.662706 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:45.662706 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:45.662706 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:45.662706 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:45.662706 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:45.662706 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 13:33:45.662706 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:33:45.662706 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 06 13:33:45.662706 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:45.662706 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 534/1058] 10.222.0.20 () {56 vars in 1105 bytes} [Wed Jul 6 13:33:45 2022] GET /identity/v3/services/image => generated 86 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:33:45.668821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5dd5bd-4d51-4f30-b962-146fbf8029a1 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:45.702227 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd019b9-e5e8-4d1a-96c0-b52d5a796e08 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:45.711191 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:45.742725 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972fca21-d10c-4a65-ad54-7e36e4e6ee76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:45.743678 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972fca21-d10c-4a65-ad54-7e36e4e6ee76 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:45.743962 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972fca21-d10c-4a65-ad54-7e36e4e6ee76 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:45.744236 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972fca21-d10c-4a65-ad54-7e36e4e6ee76 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:45.755078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972fca21-d10c-4a65-ad54-7e36e4e6ee76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:33:45.763808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-972fca21-d10c-4a65-ad54-7e36e4e6ee76 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 13:33:45.763808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:45.763808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:45.763808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:45.763808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:45.763808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:45.763808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:45.763808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:45.763808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:45.763808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:45.763808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:45.763808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:45.763808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:33:45.763808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:33:45.763808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:33:45.763808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:33:45.764768 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:33:45.764768 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:33:45.764768 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:33:45.764768 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:33:45.764768 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:45.764768 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:45.764768 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:45.764768 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:45.764768 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:45.764768 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:45.764768 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:45.764768 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:45.764768 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:45.764768 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:45.764768 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:45.764768 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:45.765711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:45.765711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:45.765711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:45.765711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:45.765711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:33:45.765711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:33:45.765711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:33:45.765711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:33:45.765711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:33:45.765711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:33:45.765711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 13:33:45.765711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:45.765711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 525/1059] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Jul 6 13:33:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:33:45.770998 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972fca21-d10c-4a65-ad54-7e36e4e6ee76 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:45.779811 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5dd5bd-4d51-4f30-b962-146fbf8029a1 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:45.817266 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16211109-3488-4bd1-b74b-afde3b424a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:45.817925 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16211109-3488-4bd1-b74b-afde3b424a3d None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:45.818263 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16211109-3488-4bd1-b74b-afde3b424a3d None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:45.818477 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16211109-3488-4bd1-b74b-afde3b424a3d None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:45.819179 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16211109-3488-4bd1-b74b-afde3b424a3d None admin] RBAC: Adding query filter params (name=image) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:45.819761 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16211109-3488-4bd1-b74b-afde3b424a3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:45.821456 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16211109-3488-4bd1-b74b-afde3b424a3d None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:45.826798 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 535/1060] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:45.832081 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16211109-3488-4bd1-b74b-afde3b424a3d None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:45.860542 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:45.878863 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972fca21-d10c-4a65-ad54-7e36e4e6ee76 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:45.913456 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d47fdf1-9ab9-45b9-890b-a10e288599f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:45.914120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47fdf1-9ab9-45b9-890b-a10e288599f7 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:45.914406 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47fdf1-9ab9-45b9-890b-a10e288599f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:45.915288 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47fdf1-9ab9-45b9-890b-a10e288599f7 None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:45.916025 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d47fdf1-9ab9-45b9-890b-a10e288599f7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:45.916326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d47fdf1-9ab9-45b9-890b-a10e288599f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:45.917705 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d47fdf1-9ab9-45b9-890b-a10e288599f7 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:45.926463 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 526/1061] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Jul 6 13:33:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:45.932466 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d47fdf1-9ab9-45b9-890b-a10e288599f7 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:45.943101 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16211109-3488-4bd1-b74b-afde3b424a3d None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:45.965485 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:45.979911 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de08e68-0e39-48af-bfe1-9267b57bfd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:45.980580 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de08e68-0e39-48af-bfe1-9267b57bfd8f None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:45.980920 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de08e68-0e39-48af-bfe1-9267b57bfd8f None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:45.981249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de08e68-0e39-48af-bfe1-9267b57bfd8f None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:45.981965 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de08e68-0e39-48af-bfe1-9267b57bfd8f None admin] RBAC: Adding query filter params (type=image) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:45.982275 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de08e68-0e39-48af-bfe1-9267b57bfd8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:45.983666 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de08e68-0e39-48af-bfe1-9267b57bfd8f None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:45.993774 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 536/1062] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:45 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:45.994906 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de08e68-0e39-48af-bfe1-9267b57bfd8f None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:46.048778 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d47fdf1-9ab9-45b9-890b-a10e288599f7 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:46.086238 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:46.088438 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7534be5-b9dc-443c-94e6-d1a87d761777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:46.089284 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7534be5-b9dc-443c-94e6-d1a87d761777 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:46.089727 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7534be5-b9dc-443c-94e6-d1a87d761777 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:46.090034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7534be5-b9dc-443c-94e6-d1a87d761777 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:46.099384 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de08e68-0e39-48af-bfe1-9267b57bfd8f None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:46.102965 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7534be5-b9dc-443c-94e6-d1a87d761777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:46.104482 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7534be5-b9dc-443c-94e6-d1a87d761777 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:46.122504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-e7534be5-b9dc-443c-94e6-d1a87d761777 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:46.122504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:46.122504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:46.122504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:46.122504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:46.122504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:46.122504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:46.122504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:46.122504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:46.122504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:46.122504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:46.122504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:46.122504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:46.122504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:46.122504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:46.123537 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:46.123537 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:46.123537 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:46.123537 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:46.123537 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:46.123537 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:46.123537 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:46.123537 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:46.123537 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:46.123537 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:46.123537 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:46.123537 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:46.123537 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:46.123537 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:46.123537 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:46.123537 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:46.123537 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:46.124432 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:46.124432 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:46.124432 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:46.124432 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:46.124432 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 527/1063] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:33:46.129851 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7534be5-b9dc-443c-94e6-d1a87d761777 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:46.136716 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb1d590-f50f-4d63-9ea6-f19d53ef8981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:46.137408 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1d590-f50f-4d63-9ea6-f19d53ef8981 None admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:46.137696 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1d590-f50f-4d63-9ea6-f19d53ef8981 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:46.137975 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1d590-f50f-4d63-9ea6-f19d53ef8981 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:46.138721 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb1d590-f50f-4d63-9ea6-f19d53ef8981 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:46.140083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb1d590-f50f-4d63-9ea6-f19d53ef8981 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:46.161979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 537/1064] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Jul 6 13:33:45 2022] POST /identity/v3/endpoints => generated 327 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:33:46.235835 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:46.258816 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7534be5-b9dc-443c-94e6-d1a87d761777 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:46.309220 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8033dc6-3896-46b1-9c13-40620fea0b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:46.309891 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8033dc6-3896-46b1-9c13-40620fea0b39 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:46.310172 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8033dc6-3896-46b1-9c13-40620fea0b39 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:46.310444 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8033dc6-3896-46b1-9c13-40620fea0b39 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:46.311163 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8033dc6-3896-46b1-9c13-40620fea0b39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:46.311467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8033dc6-3896-46b1-9c13-40620fea0b39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:46.312818 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8033dc6-3896-46b1-9c13-40620fea0b39 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:46.327597 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 528/1065] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:46.333547 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb1d590-f50f-4d63-9ea6-f19d53ef8981 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:46.440491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb1d590-f50f-4d63-9ea6-f19d53ef8981 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:46.474261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8af96-7dab-4e50-961f-7884f2a1c2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:46.475058 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8af96-7dab-4e50-961f-7884f2a1c2e7 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:46.475340 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8af96-7dab-4e50-961f-7884f2a1c2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:46.475614 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8af96-7dab-4e50-961f-7884f2a1c2e7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:46.484474 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a8af96-7dab-4e50-961f-7884f2a1c2e7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:46.485948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a8af96-7dab-4e50-961f-7884f2a1c2e7 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:46.501066 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-19a8af96-7dab-4e50-961f-7884f2a1c2e7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 06 13:33:46.501066 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:46.501066 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:46.501066 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:46.501066 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:46.501066 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:46.501066 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:46.501066 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:46.501066 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:46.501066 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:46.501066 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:46.501066 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:46.501066 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:33:46.501066 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:33:46.501066 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:33:46.501066 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:33:46.502249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:46.502249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:46.502249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:33:46.502249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:46.502249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:33:46.502249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:46.502249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:46.502249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:46.502249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:46.502249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:46.502249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:46.502249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:46.502249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:46.502249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:46.502249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:46.502249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:46.502249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:46.502249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:46.503238 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:46.503238 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:46.503238 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:33:46.503238 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:33:46.503238 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:33:46.503238 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:33:46.503238 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:33:46.503238 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:33:46.503238 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 06 13:33:46.503238 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:46.503238 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 538/1066] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Jul 6 13:33:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:46.508287 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8033dc6-3896-46b1-9c13-40620fea0b39 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:46.617588 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8033dc6-3896-46b1-9c13-40620fea0b39 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:46.619369 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68263eab-f72f-43d0-9b58-65aa35443bdd None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:46.619662 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68263eab-f72f-43d0-9b58-65aa35443bdd None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:46.619946 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68263eab-f72f-43d0-9b58-65aa35443bdd None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:46.621834 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 539/1067] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:33:46.629020 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707572d6-88c2-4de4-8d5d-cb71872ae27c None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:46.629328 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707572d6-88c2-4de4-8d5d-cb71872ae27c None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:46.629614 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707572d6-88c2-4de4-8d5d-cb71872ae27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:46.636956 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:46.648714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-707572d6-88c2-4de4-8d5d-cb71872ae27c None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:46.650321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-707572d6-88c2-4de4-8d5d-cb71872ae27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:46.659474 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176171f0-a3a7-4a4b-a03d-954bb219a53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:46.660218 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176171f0-a3a7-4a4b-a03d-954bb219a53a None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:46.660738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176171f0-a3a7-4a4b-a03d-954bb219a53a None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:46.661037 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176171f0-a3a7-4a4b-a03d-954bb219a53a None admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:46.661914 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176171f0-a3a7-4a4b-a03d-954bb219a53a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:46.662225 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176171f0-a3a7-4a4b-a03d-954bb219a53a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:46.663659 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176171f0-a3a7-4a4b-a03d-954bb219a53a None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:46.668117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:46.693381 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 529/1068] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Jul 6 13:33:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:46.694178 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176171f0-a3a7-4a4b-a03d-954bb219a53a None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:46.738607 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 540/1069] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 13:33:46.769644 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707572d6-88c2-4de4-8d5d-cb71872ae27c None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:46.814577 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176171f0-a3a7-4a4b-a03d-954bb219a53a None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:46.852182 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e384957-3ae2-4589-80cb-4ed0a40051cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:46.853633 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e384957-3ae2-4589-80cb-4ed0a40051cb None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:46.853981 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e384957-3ae2-4589-80cb-4ed0a40051cb None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:46.854296 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e384957-3ae2-4589-80cb-4ed0a40051cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:46.865596 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e384957-3ae2-4589-80cb-4ed0a40051cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:46.867169 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e384957-3ae2-4589-80cb-4ed0a40051cb None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:46.877772 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707572d6-88c2-4de4-8d5d-cb71872ae27c None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:46.895086 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-8e384957-3ae2-4589-80cb-4ed0a40051cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:46.895086 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:46.895086 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:46.895086 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:46.895086 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:46.895086 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:46.895086 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:46.895086 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:46.895086 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:46.895086 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:46.895086 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:46.895086 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:46.895086 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:46.895086 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:46.895086 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:46.896047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:46.896047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:46.896047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:46.896047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:46.896047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:46.896047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:46.896047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:46.896047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:46.896047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:46.896047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:46.896047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:46.896047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:46.896047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:46.896047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:46.896047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:46.896047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:46.896047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:46.897258 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:46.897258 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:46.897258 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:46.897258 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:46.897258 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 530/1070] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:46.903910 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e384957-3ae2-4589-80cb-4ed0a40051cb None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:46.919590 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4574cd-cd94-4f6d-97f6-b1f7f0e41c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:46.920392 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4574cd-cd94-4f6d-97f6-b1f7f0e41c0f None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:46.920677 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4574cd-cd94-4f6d-97f6-b1f7f0e41c0f None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:46.921035 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4574cd-cd94-4f6d-97f6-b1f7f0e41c0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:46.922869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:46.934206 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4574cd-cd94-4f6d-97f6-b1f7f0e41c0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:33:46.942660 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-7a4574cd-cd94-4f6d-97f6-b1f7f0e41c0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:33:46.942660 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:46.942660 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:46.942660 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:46.942660 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:46.942660 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:46.942660 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:46.942660 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:46.942660 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:46.942660 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:46.942660 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:46.942660 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:46.942660 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:33:46.942660 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:33:46.942660 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:33:46.942660 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:33:46.943701 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:33:46.943701 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:33:46.943701 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:33:46.943701 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:33:46.943701 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:46.943701 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:46.943701 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:46.943701 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:46.943701 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:46.943701 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:46.943701 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:46.943701 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:46.943701 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:46.943701 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:46.943701 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:46.943701 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:46.944622 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:46.944622 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:46.944622 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:46.944622 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:46.944622 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:33:46.944622 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:33:46.944622 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:33:46.944622 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:33:46.944622 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:33:46.944622 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:33:46.944622 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:33:46.944622 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:46.944622 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 541/1071] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:33:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:46.950032 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4574cd-cd94-4f6d-97f6-b1f7f0e41c0f None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:46.975737 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:47.038090 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e384957-3ae2-4589-80cb-4ed0a40051cb None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:47.065061 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4574cd-cd94-4f6d-97f6-b1f7f0e41c0f None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:47.075175 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50dcb9-3609-4230-8f51-675a5f342e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:47.075845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50dcb9-3609-4230-8f51-675a5f342e2e None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:47.076139 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50dcb9-3609-4230-8f51-675a5f342e2e None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:47.076418 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50dcb9-3609-4230-8f51-675a5f342e2e None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:47.077176 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50dcb9-3609-4230-8f51-675a5f342e2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:47.077490 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50dcb9-3609-4230-8f51-675a5f342e2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:47.078973 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50dcb9-3609-4230-8f51-675a5f342e2e None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:47.095644 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 531/1072] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:47.096647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50dcb9-3609-4230-8f51-675a5f342e2e None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:47.110226 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c19283-7b66-4821-b294-e2060cc5f770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:47.110940 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c19283-7b66-4821-b294-e2060cc5f770 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:47.111234 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c19283-7b66-4821-b294-e2060cc5f770 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:47.111514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c19283-7b66-4821-b294-e2060cc5f770 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:47.112262 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c19283-7b66-4821-b294-e2060cc5f770 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:47.112570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c19283-7b66-4821-b294-e2060cc5f770 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:47.114014 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c19283-7b66-4821-b294-e2060cc5f770 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:47.124103 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 542/1073] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:33:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:47.129976 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c19283-7b66-4821-b294-e2060cc5f770 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:47.185182 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:47.214469 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50dcb9-3609-4230-8f51-675a5f342e2e None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:47.222367 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:47.256907 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c19283-7b66-4821-b294-e2060cc5f770 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:47.262985 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94124d08-5bf2-4da9-9552-18c1e5f36182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:47.263872 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94124d08-5bf2-4da9-9552-18c1e5f36182 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:47.264233 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94124d08-5bf2-4da9-9552-18c1e5f36182 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:47.264603 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94124d08-5bf2-4da9-9552-18c1e5f36182 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:47.273291 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94124d08-5bf2-4da9-9552-18c1e5f36182 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:47.275553 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94124d08-5bf2-4da9-9552-18c1e5f36182 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:47.289214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-94124d08-5bf2-4da9-9552-18c1e5f36182 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:47.289214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:47.289214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:47.289214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:47.289214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:47.289214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:47.289214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:47.289214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:47.289214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:47.289214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:47.289214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:47.289214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:47.289214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:33:47.289214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:33:47.289214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:33:47.290437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:33:47.290437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:47.290437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:47.290437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:47.290437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:47.290437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:47.290437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:47.290437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:47.290437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:47.290437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:47.290437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:47.290437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:47.290437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:47.290437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:47.290437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:47.290437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:47.290437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:47.291503 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:33:47.291503 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:33:47.291503 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:33:47.291503 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:33:47.291503 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:33:47.291503 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:33:47.291503 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:47.291503 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:47.295866 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 532/1074] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:33:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:47.296601 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94124d08-5bf2-4da9-9552-18c1e5f36182 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:47.306583 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd8b76-0281-4b37-b595-fea76fa31c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:47.307437 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd8b76-0281-4b37-b595-fea76fa31c57 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:47.307721 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd8b76-0281-4b37-b595-fea76fa31c57 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:47.307997 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd8b76-0281-4b37-b595-fea76fa31c57 None admin] PATH_INFO: `/v3/users/3e2a815b683e465e91713f548a64b13f` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:47.309355 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcd8b76-0281-4b37-b595-fea76fa31c57 None admin] RBAC: Authorizing `identity:get_user(user_id=3e2a815b683e465e91713f548a64b13f)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:47.310901 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcd8b76-0281-4b37-b595-fea76fa31c57 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:47.318047 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 543/1075] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:33:47 2022] GET /identity/v3/users/3e2a815b683e465e91713f548a64b13f => generated 285 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:47.318417 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd8b76-0281-4b37-b595-fea76fa31c57 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:47.413267 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94124d08-5bf2-4da9-9552-18c1e5f36182 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:47.427191 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd8b76-0281-4b37-b595-fea76fa31c57 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:47.452957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:47.454288 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f071f-c62a-4cde-8350-3fa0ccaa34a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:47.455011 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f071f-c62a-4cde-8350-3fa0ccaa34a2 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:47.455426 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f071f-c62a-4cde-8350-3fa0ccaa34a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:47.455582 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f071f-c62a-4cde-8350-3fa0ccaa34a2 None admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:47.456372 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f071f-c62a-4cde-8350-3fa0ccaa34a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:47.456805 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f071f-c62a-4cde-8350-3fa0ccaa34a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:47.458165 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f071f-c62a-4cde-8350-3fa0ccaa34a2 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:47.465690 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ecf6b1-301c-41bf-bfb8-00a631f52d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:47.467201 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ecf6b1-301c-41bf-bfb8-00a631f52d8b None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:47.467497 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ecf6b1-301c-41bf-bfb8-00a631f52d8b None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:47.467779 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ecf6b1-301c-41bf-bfb8-00a631f52d8b None admin] PATH_INFO: `/v3/projects/9fa3428bf0d3459a8e7e86d666e373e6` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:47.469143 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ecf6b1-301c-41bf-bfb8-00a631f52d8b None admin] RBAC: Authorizing `identity:get_project(project_id=9fa3428bf0d3459a8e7e86d666e373e6)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:47.474844 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 533/1076] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:33:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:47.474973 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ecf6b1-301c-41bf-bfb8-00a631f52d8b None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:47.474973 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 544/1077] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:33:47 2022] GET /identity/v3/projects/9fa3428bf0d3459a8e7e86d666e373e6 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:47.478034 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ecf6b1-301c-41bf-bfb8-00a631f52d8b None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:47.478181 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f071f-c62a-4cde-8350-3fa0ccaa34a2 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:47.511592 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:47.593869 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f071f-c62a-4cde-8350-3fa0ccaa34a2 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:47.596382 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ecf6b1-301c-41bf-bfb8-00a631f52d8b None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:47.635749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33acf2-f860-4ea1-a04c-743f85c5b671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:47.636965 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33acf2-f860-4ea1-a04c-743f85c5b671 None admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:47.637449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33acf2-f860-4ea1-a04c-743f85c5b671 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:47.637901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33acf2-f860-4ea1-a04c-743f85c5b671 None admin] PATH_INFO: `/v3/projects/bfe6c3d8da2743be9609184db2506303/users/6647eb1e8a1544ef9d3a9806f2d1fc51/roles/8d259fab1ed64ced91f18487e90ee097` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:47.640437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b33acf2-f860-4ea1-a04c-743f85c5b671 None admin] RBAC: Authorizing `identity:create_grant(project_id=bfe6c3d8da2743be9609184db2506303, user_id=6647eb1e8a1544ef9d3a9806f2d1fc51, role_id=8d259fab1ed64ced91f18487e90ee097)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:47.642185 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b33acf2-f860-4ea1-a04c-743f85c5b671 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:47.642580 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865783ed-e721-4481-ae43-57b173611348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:47.643463 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865783ed-e721-4481-ae43-57b173611348 None admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:47.643747 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865783ed-e721-4481-ae43-57b173611348 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:47.644055 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865783ed-e721-4481-ae43-57b173611348 None admin] PATH_INFO: `/v3/projects/9fa3428bf0d3459a8e7e86d666e373e6/users/3e2a815b683e465e91713f548a64b13f/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:47.646069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865783ed-e721-4481-ae43-57b173611348 None admin] RBAC: Authorizing `identity:create_grant(project_id=9fa3428bf0d3459a8e7e86d666e373e6, user_id=3e2a815b683e465e91713f548a64b13f, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:47.647471 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865783ed-e721-4481-ae43-57b173611348 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:47.669053 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 534/1078] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jul 6 13:33:47 2022] PUT /identity/v3/projects/bfe6c3d8da2743be9609184db2506303/users/6647eb1e8a1544ef9d3a9806f2d1fc51/roles/8d259fab1ed64ced91f18487e90ee097 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:33:47.669283 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 545/1079] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jul 6 13:33:47 2022] PUT /identity/v3/projects/9fa3428bf0d3459a8e7e86d666e373e6/users/3e2a815b683e465e91713f548a64b13f/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:33:48.310724 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34baf38-e584-4269-a5bd-abeab7a594ce None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:48.311008 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34baf38-e584-4269-a5bd-abeab7a594ce None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:48.311288 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34baf38-e584-4269-a5bd-abeab7a594ce None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:48.312690 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 535/1080] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:48.317655 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd19f40-5113-4c25-bc16-998f38b36c4a None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:48.317945 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd19f40-5113-4c25-bc16-998f38b36c4a None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:48.318256 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd19f40-5113-4c25-bc16-998f38b36c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:48.335910 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-6fd19f40-5113-4c25-bc16-998f38b36c4a None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:48.338055 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-6fd19f40-5113-4c25-bc16-998f38b36c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:48.386846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:48.408975 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 546/1081] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:48.431674 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34baf38-e584-4269-a5bd-abeab7a594ce None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:48.518295 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:48.537909 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34baf38-e584-4269-a5bd-abeab7a594ce None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:48.574672 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a88ed-70f4-4cfd-afce-fe7a652f38a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:48.575327 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a88ed-70f4-4cfd-afce-fe7a652f38a8 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:48.575613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a88ed-70f4-4cfd-afce-fe7a652f38a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:48.575884 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a88ed-70f4-4cfd-afce-fe7a652f38a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:48.644031 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5a88ed-70f4-4cfd-afce-fe7a652f38a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:48.645597 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5a88ed-70f4-4cfd-afce-fe7a652f38a8 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:48.728375 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:48.745830 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 536/1082] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:33:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 13:33:48.751558 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd19f40-5113-4c25-bc16-998f38b36c4a None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:48.868064 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd19f40-5113-4c25-bc16-998f38b36c4a None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:48.918010 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8153d-4457-45a1-a8ca-9093e105f6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:48.918802 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8153d-4457-45a1-a8ca-9093e105f6f9 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:48.919078 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8153d-4457-45a1-a8ca-9093e105f6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:48.919348 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8153d-4457-45a1-a8ca-9093e105f6f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:48.929503 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc8153d-4457-45a1-a8ca-9093e105f6f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:48.930942 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc8153d-4457-45a1-a8ca-9093e105f6f9 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:48.948325 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-5cc8153d-4457-45a1-a8ca-9093e105f6f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:48.948325 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:48.948325 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:48.948325 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:48.948325 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:48.948325 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:48.948325 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:48.948325 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:48.948325 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:48.948325 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:48.948325 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:48.948325 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:48.948325 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:48.948325 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:48.948325 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:48.949499 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:48.949499 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:48.949499 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:48.949499 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:48.949499 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:48.949499 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:48.949499 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:48.949499 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:48.949499 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:48.949499 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:48.949499 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:48.949499 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:48.949499 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:48.949499 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:48.949499 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:48.949499 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:48.949499 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:48.955232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:48.955232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:48.955232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:48.955232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:48.955232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 547/1083] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:48.955527 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a88ed-70f4-4cfd-afce-fe7a652f38a8 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:49.053924 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a88ed-70f4-4cfd-afce-fe7a652f38a8 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:49.086923 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b19400-e4b0-47c2-ab84-438a76a7edf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:49.087577 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b19400-e4b0-47c2-ab84-438a76a7edf2 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:49.087855 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b19400-e4b0-47c2-ab84-438a76a7edf2 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:49.088125 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b19400-e4b0-47c2-ab84-438a76a7edf2 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:49.088835 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b19400-e4b0-47c2-ab84-438a76a7edf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:49.089169 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b19400-e4b0-47c2-ab84-438a76a7edf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:49.090520 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b19400-e4b0-47c2-ab84-438a76a7edf2 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:49.108306 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 537/1084] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:49.728440 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827aca2-2060-4ad7-9d50-dd5e5c5696c7 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:49.728649 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827aca2-2060-4ad7-9d50-dd5e5c5696c7 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:49.728924 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827aca2-2060-4ad7-9d50-dd5e5c5696c7 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:49.729951 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acf1f7-5576-4062-b624-e452f59bd9bf None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:49.730366 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acf1f7-5576-4062-b624-e452f59bd9bf None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:49.730439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 548/1085] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:49.730566 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acf1f7-5576-4062-b624-e452f59bd9bf None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:49.732010 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 538/1086] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:33:49.735157 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d399d-399e-40ce-ba2d-161123c43674 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:49.735444 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d399d-399e-40ce-ba2d-161123c43674 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:49.735733 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d399d-399e-40ce-ba2d-161123c43674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:49.736887 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05f1c9-9cfe-48f2-93d8-34cb21abd968 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:49.737288 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05f1c9-9cfe-48f2-93d8-34cb21abd968 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:49.737576 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05f1c9-9cfe-48f2-93d8-34cb21abd968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:49.752819 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-ba4d399d-399e-40ce-ba2d-161123c43674 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:49.754016 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-ba4d399d-399e-40ce-ba2d-161123c43674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:49.754431 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-ee05f1c9-9cfe-48f2-93d8-34cb21abd968 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:49.755588 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-ee05f1c9-9cfe-48f2-93d8-34cb21abd968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:49.779962 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:49.799501 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:49.820278 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 549/1087] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:49.823722 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 539/1088] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 13:33:49.844248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d399d-399e-40ce-ba2d-161123c43674 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:49.847822 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee05f1c9-9cfe-48f2-93d8-34cb21abd968 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:49.942439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d399d-399e-40ce-ba2d-161123c43674 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:49.947634 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee05f1c9-9cfe-48f2-93d8-34cb21abd968 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:49.976869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f933982-bf9d-4532-a8ae-bd8577ca8e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:49.978836 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f933982-bf9d-4532-a8ae-bd8577ca8e28 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:49.979065 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f933982-bf9d-4532-a8ae-bd8577ca8e28 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:49.979351 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f933982-bf9d-4532-a8ae-bd8577ca8e28 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:49.982551 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8098a-92a1-4e8f-aed4-5c54bdce5524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:49.983353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8098a-92a1-4e8f-aed4-5c54bdce5524 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:49.983634 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8098a-92a1-4e8f-aed4-5c54bdce5524 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:49.983909 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8098a-92a1-4e8f-aed4-5c54bdce5524 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:49.989372 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f933982-bf9d-4532-a8ae-bd8577ca8e28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:33:49.995006 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e8098a-92a1-4e8f-aed4-5c54bdce5524 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:33:49.998003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-6f933982-bf9d-4532-a8ae-bd8577ca8e28 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:33:49.998003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:49.998003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:49.998003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:49.998003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:49.998003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:49.998003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:49.998003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:49.998003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:49.998003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:49.998003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:49.998003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:49.998003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:33:49.998003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:33:49.998003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:33:49.998003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:33:49.998982 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:33:49.998982 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:33:49.998982 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:33:49.998982 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:33:49.998982 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:49.998982 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:49.998982 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:49.998982 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:49.998982 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:49.998982 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:49.998982 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:49.998982 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:49.998982 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:49.998982 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:49.998982 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:49.998982 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:49.999857 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:49.999857 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:49.999857 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:49.999857 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:49.999857 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:33:49.999857 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:33:49.999857 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:33:49.999857 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:33:49.999857 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:33:49.999857 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:33:49.999857 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:33:49.999857 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:49.999857 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 550/1089] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:33:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:50.003410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-05e8098a-92a1-4e8f-aed4-5c54bdce5524 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 13:33:50.003410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:50.003410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:50.003410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:50.003410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:50.003410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:50.003410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:50.003410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:50.003410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:50.003410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:50.003410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:50.003410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:50.003410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:33:50.003410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:33:50.003410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:33:50.003410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:33:50.004411 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:33:50.004411 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:33:50.004411 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:33:50.004411 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:33:50.004411 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:50.004411 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:50.004411 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:50.004411 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:50.004411 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:50.004411 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:50.004411 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:50.004411 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:50.004411 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:50.004411 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:50.004411 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:50.004411 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:50.005307 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:50.005307 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:50.005307 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:50.005307 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:50.005307 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:33:50.005307 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:33:50.005307 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:33:50.005307 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:33:50.005307 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:33:50.005307 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:33:50.005307 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 13:33:50.005307 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:50.006542 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f933982-bf9d-4532-a8ae-bd8577ca8e28 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:50.006628 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 540/1090] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Jul 6 13:33:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:33:50.010391 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8098a-92a1-4e8f-aed4-5c54bdce5524 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:50.030537 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:50.045759 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:50.105570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f933982-bf9d-4532-a8ae-bd8577ca8e28 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:50.108487 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8098a-92a1-4e8f-aed4-5c54bdce5524 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:50.138855 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d7b0e-f48b-4ef8-a56d-a4930217a077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:50.139516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d7b0e-f48b-4ef8-a56d-a4930217a077 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:50.139796 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d7b0e-f48b-4ef8-a56d-a4930217a077 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:50.140070 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d7b0e-f48b-4ef8-a56d-a4930217a077 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:50.140808 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d7b0e-f48b-4ef8-a56d-a4930217a077 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:50.141144 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d7b0e-f48b-4ef8-a56d-a4930217a077 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:50.142202 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c12698-353e-49ef-afc3-2c4c495e7fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:50.142496 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d7b0e-f48b-4ef8-a56d-a4930217a077 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:50.142861 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c12698-353e-49ef-afc3-2c4c495e7fbc None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:50.143147 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c12698-353e-49ef-afc3-2c4c495e7fbc None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:50.143424 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c12698-353e-49ef-afc3-2c4c495e7fbc None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:50.144201 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c12698-353e-49ef-afc3-2c4c495e7fbc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:50.144518 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c12698-353e-49ef-afc3-2c4c495e7fbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:50.145905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c12698-353e-49ef-afc3-2c4c495e7fbc None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:50.150772 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 551/1091] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:33:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:50.154243 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 541/1092] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Jul 6 13:33:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:50.156160 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d7b0e-f48b-4ef8-a56d-a4930217a077 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:50.159509 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c12698-353e-49ef-afc3-2c4c495e7fbc None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:50.239572 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:50.254357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:50.254919 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d7b0e-f48b-4ef8-a56d-a4930217a077 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:50.258323 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c12698-353e-49ef-afc3-2c4c495e7fbc None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:50.288282 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e38ca5-a043-4c3e-8616-4efec9ffd236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:50.289187 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e38ca5-a043-4c3e-8616-4efec9ffd236 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:50.293084 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e38ca5-a043-4c3e-8616-4efec9ffd236 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:50.293084 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e38ca5-a043-4c3e-8616-4efec9ffd236 None admin] PATH_INFO: `/v3/users/3e2a815b683e465e91713f548a64b13f` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:50.293084 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e38ca5-a043-4c3e-8616-4efec9ffd236 None admin] RBAC: Authorizing `identity:get_user(user_id=3e2a815b683e465e91713f548a64b13f)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:50.293377 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e38ca5-a043-4c3e-8616-4efec9ffd236 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:50.295708 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 552/1093] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:33:50 2022] GET /identity/v3/users/3e2a815b683e465e91713f548a64b13f => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:50.295801 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93454304-74a9-439e-ad78-a7ff15626960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:50.296292 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93454304-74a9-439e-ad78-a7ff15626960 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:50.296575 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93454304-74a9-439e-ad78-a7ff15626960 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:50.296849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93454304-74a9-439e-ad78-a7ff15626960 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:50.300869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e38ca5-a043-4c3e-8616-4efec9ffd236 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:50.311918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93454304-74a9-439e-ad78-a7ff15626960 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:50.313390 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93454304-74a9-439e-ad78-a7ff15626960 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:50.331091 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-93454304-74a9-439e-ad78-a7ff15626960 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:50.331091 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:50.331091 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:50.331091 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:50.331091 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:50.331091 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:50.331091 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:50.331091 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:50.331091 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:50.331091 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:50.331091 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:50.331091 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:50.331091 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:50.331091 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:50.331091 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:50.332173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:50.332173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:50.332173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:50.332173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:50.332173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:50.332173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:50.332173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:50.332173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:50.332173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:50.332173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:50.332173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:50.332173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:50.332173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:50.332173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:50.332173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:50.332173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:50.332173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:50.333136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:50.333136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:50.333136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:50.333136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:50.333136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 542/1094] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:33:50.338026 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93454304-74a9-439e-ad78-a7ff15626960 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:50.403198 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e38ca5-a043-4c3e-8616-4efec9ffd236 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:50.436230 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b872181-ff10-4b85-b557-0401b4d01eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:50.436406 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93454304-74a9-439e-ad78-a7ff15626960 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:50.436871 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b872181-ff10-4b85-b557-0401b4d01eed None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:50.437192 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b872181-ff10-4b85-b557-0401b4d01eed None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:50.437471 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b872181-ff10-4b85-b557-0401b4d01eed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:50.460883 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:50.469644 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097c214-0237-4f5a-8da3-e0be3d2042f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:50.470299 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097c214-0237-4f5a-8da3-e0be3d2042f4 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:50.470585 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097c214-0237-4f5a-8da3-e0be3d2042f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:50.470865 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097c214-0237-4f5a-8da3-e0be3d2042f4 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:50.471584 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5097c214-0237-4f5a-8da3-e0be3d2042f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:50.471894 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5097c214-0237-4f5a-8da3-e0be3d2042f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:50.473273 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5097c214-0237-4f5a-8da3-e0be3d2042f4 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:50.484881 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 543/1095] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:50.490372 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097c214-0237-4f5a-8da3-e0be3d2042f4 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:50.504755 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b872181-ff10-4b85-b557-0401b4d01eed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:50.504918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:50.506526 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b872181-ff10-4b85-b557-0401b4d01eed None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:50.593319 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097c214-0237-4f5a-8da3-e0be3d2042f4 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:50.604100 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 553/1096] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:33:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 06 13:33:50.609832 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b872181-ff10-4b85-b557-0401b4d01eed None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:50.631990 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ec850-9560-4075-8df8-ae4c6a4a57fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:50.632889 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ec850-9560-4075-8df8-ae4c6a4a57fa None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:50.633431 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ec850-9560-4075-8df8-ae4c6a4a57fa None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:50.633710 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ec850-9560-4075-8df8-ae4c6a4a57fa None admin] PATH_INFO: `/v3/users/nova` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:50.641006 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6ec850-9560-4075-8df8-ae4c6a4a57fa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:50.642469 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6ec850-9560-4075-8df8-ae4c6a4a57fa None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:50.657407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-3b6ec850-9560-4075-8df8-ae4c6a4a57fa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 06 13:33:50.657407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:50.657407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:50.657407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:50.657407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:50.657407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:50.657407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:50.657407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:50.657407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:50.657407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:50.657407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:50.657407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:50.657407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:33:50.657407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:33:50.657407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:33:50.657407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:33:50.658499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:50.658499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:50.658499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:33:50.658499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:50.658499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:33:50.658499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:33:50.658499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:50.658499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:50.658499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:50.658499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:50.658499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:50.658499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:50.658499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:50.658499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:50.658499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:50.658499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:50.658499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:50.658499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:50.659531 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:50.659531 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:50.659531 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:33:50.659531 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:33:50.659531 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:33:50.659531 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:33:50.659531 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:33:50.659531 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:33:50.659531 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 06 13:33:50.659531 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:33:50.659531 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 544/1097] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Jul 6 13:33:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:33:50.664142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ec850-9560-4075-8df8-ae4c6a4a57fa None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:50.667077 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:50.711194 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b872181-ff10-4b85-b557-0401b4d01eed None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:50.736699 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:50.745391 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dfad44-07d2-4801-b295-40c0faf2dc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:50.746247 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dfad44-07d2-4801-b295-40c0faf2dc5b None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:50.746530 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dfad44-07d2-4801-b295-40c0faf2dc5b None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:50.746803 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dfad44-07d2-4801-b295-40c0faf2dc5b None admin] PATH_INFO: `/v3/projects/9fa3428bf0d3459a8e7e86d666e373e6` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:50.747966 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dfad44-07d2-4801-b295-40c0faf2dc5b None admin] RBAC: Authorizing `identity:get_project(project_id=9fa3428bf0d3459a8e7e86d666e373e6)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:50.749415 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dfad44-07d2-4801-b295-40c0faf2dc5b None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:50.751101 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 554/1098] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:33:50 2022] GET /identity/v3/projects/9fa3428bf0d3459a8e7e86d666e373e6 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:50.756876 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dfad44-07d2-4801-b295-40c0faf2dc5b None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:50.772568 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ec850-9560-4075-8df8-ae4c6a4a57fa None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:50.807149 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7394298-6a32-4ef4-9660-f887d6206dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:50.807811 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7394298-6a32-4ef4-9660-f887d6206dd6 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:50.808093 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7394298-6a32-4ef4-9660-f887d6206dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:50.808369 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7394298-6a32-4ef4-9660-f887d6206dd6 None admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:50.809240 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7394298-6a32-4ef4-9660-f887d6206dd6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:50.809551 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7394298-6a32-4ef4-9660-f887d6206dd6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:50.810976 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7394298-6a32-4ef4-9660-f887d6206dd6 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:50.828216 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 545/1099] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Jul 6 13:33:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:50.833710 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7394298-6a32-4ef4-9660-f887d6206dd6 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:50.855315 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dfad44-07d2-4801-b295-40c0faf2dc5b None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:50.876663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:50.888828 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1733e5-3045-4538-b0fa-b4455639ae4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:50.889496 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1733e5-3045-4538-b0fa-b4455639ae4b None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:50.889775 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1733e5-3045-4538-b0fa-b4455639ae4b None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:50.890048 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1733e5-3045-4538-b0fa-b4455639ae4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:50.890867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1733e5-3045-4538-b0fa-b4455639ae4b None admin] RBAC: Adding query filter params (role.id=795daa2c78354fd48e4a741e1182cc98, scope.project.id=9fa3428bf0d3459a8e7e86d666e373e6, user.id=3e2a815b683e465e91713f548a64b13f) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:50.891172 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1733e5-3045-4538-b0fa-b4455639ae4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:50.892573 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1733e5-3045-4538-b0fa-b4455639ae4b None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:50.898336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 555/1100] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:33:50 2022] GET /identity/v3/role_assignments?user.id=3e2a815b683e465e91713f548a64b13f&scope.project.id=9fa3428bf0d3459a8e7e86d666e373e6&role.id=795daa2c78354fd48e4a741e1182cc98 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:33:50.934601 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7394298-6a32-4ef4-9660-f887d6206dd6 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:50.964108 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:50.973380 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03f9da-9f64-4c0d-9025-dd153a8dcfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:50.974045 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03f9da-9f64-4c0d-9025-dd153a8dcfb0 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:50.974330 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03f9da-9f64-4c0d-9025-dd153a8dcfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:50.974605 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03f9da-9f64-4c0d-9025-dd153a8dcfb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:51.048128 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03f9da-9f64-4c0d-9025-dd153a8dcfb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:51.049347 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03f9da-9f64-4c0d-9025-dd153a8dcfb0 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:51.144849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 546/1101] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:33:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 06 13:33:51.151543 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1733e5-3045-4538-b0fa-b4455639ae4b None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:51.252547 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1733e5-3045-4538-b0fa-b4455639ae4b None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:51.285777 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b9b72-0d43-4129-96b6-cb8920732ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:51.286560 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b9b72-0d43-4129-96b6-cb8920732ab7 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:51.286841 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b9b72-0d43-4129-96b6-cb8920732ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:51.287117 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b9b72-0d43-4129-96b6-cb8920732ab7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:51.299050 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b9b72-0d43-4129-96b6-cb8920732ab7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:51.300297 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b9b72-0d43-4129-96b6-cb8920732ab7 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:51.317167 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-d05b9b72-0d43-4129-96b6-cb8920732ab7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:51.317167 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:51.317167 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:51.317167 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:51.317167 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:51.317167 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:51.317167 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:51.317167 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:51.317167 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:51.317167 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:51.317167 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:51.317167 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:51.317167 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:33:51.317167 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:33:51.317167 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:33:51.318286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:33:51.318286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:51.318286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:51.318286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:51.318286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:51.318286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:51.318286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:51.318286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:51.318286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:51.318286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:51.318286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:51.318286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:51.318286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:51.318286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:51.318286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:51.318286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:51.318286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:51.319543 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:33:51.319543 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:33:51.319543 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:33:51.319543 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:51.321374 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 556/1102] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:51.324028 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03f9da-9f64-4c0d-9025-dd153a8dcfb0 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:51.351346 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:51.422727 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03f9da-9f64-4c0d-9025-dd153a8dcfb0 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:51.457111 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ad2cb-3921-4ab3-b963-d8d2c1060652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:51.457770 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ad2cb-3921-4ab3-b963-d8d2c1060652 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:51.458052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ad2cb-3921-4ab3-b963-d8d2c1060652 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:51.458327 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ad2cb-3921-4ab3-b963-d8d2c1060652 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:51.459045 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900ad2cb-3921-4ab3-b963-d8d2c1060652 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:51.459352 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900ad2cb-3921-4ab3-b963-d8d2c1060652 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:51.460697 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900ad2cb-3921-4ab3-b963-d8d2c1060652 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:51.472427 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 547/1103] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:51.477950 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b9b72-0d43-4129-96b6-cb8920732ab7 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:51.517514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:51.579094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b9b72-0d43-4129-96b6-cb8920732ab7 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:51.611728 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48651d19-765e-4d74-ae3f-95198f65f03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:51.612513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48651d19-765e-4d74-ae3f-95198f65f03c None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:51.612795 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48651d19-765e-4d74-ae3f-95198f65f03c None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:51.613097 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48651d19-765e-4d74-ae3f-95198f65f03c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:51.619927 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48651d19-765e-4d74-ae3f-95198f65f03c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:51.621385 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48651d19-765e-4d74-ae3f-95198f65f03c None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:51.635801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-48651d19-765e-4d74-ae3f-95198f65f03c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:51.635801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:33:51.635801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:33:51.635801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:33:51.635801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:33:51.635801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:33:51.635801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:33:51.635801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:33:51.635801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:33:51.635801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:33:51.635801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:33:51.635801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:33:51.635801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:33:51.635801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:33:51.635801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:33:51.636699 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:33:51.636699 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:33:51.636699 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:33:51.636699 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:33:51.636699 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:33:51.636699 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:33:51.636699 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:33:51.636699 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:33:51.636699 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:33:51.636699 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:33:51.636699 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:33:51.636699 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:33:51.636699 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:33:51.636699 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:33:51.636699 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:33:51.636699 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:33:51.636699 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:33:51.637968 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:33:51.637968 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:33:51.637968 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:33:51.637968 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:33:51.637968 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:33:51.637968 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:33:51.637968 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:33:51.637968 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:33:51.637968 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 557/1104] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:33:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:33:51.642631 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ad2cb-3921-4ab3-b963-d8d2c1060652 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:51.731797 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:51.744904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ad2cb-3921-4ab3-b963-d8d2c1060652 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:51.780198 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7d155-3066-49c7-8179-2fb368909056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:51.780848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7d155-3066-49c7-8179-2fb368909056 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:51.781166 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7d155-3066-49c7-8179-2fb368909056 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:51.781445 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7d155-3066-49c7-8179-2fb368909056 None admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:51.782237 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7d155-3066-49c7-8179-2fb368909056 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:51.782547 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7d155-3066-49c7-8179-2fb368909056 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:51.783972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7d155-3066-49c7-8179-2fb368909056 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:51.796681 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 548/1105] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:33:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:51.802864 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48651d19-765e-4d74-ae3f-95198f65f03c None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:51.898386 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:51.903083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48651d19-765e-4d74-ae3f-95198f65f03c None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:51.939376 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be97163-d589-421d-a390-ac7582d25a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:51.940060 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be97163-d589-421d-a390-ac7582d25a2a None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:51.940345 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be97163-d589-421d-a390-ac7582d25a2a None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:51.940623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be97163-d589-421d-a390-ac7582d25a2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:51.941568 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be97163-d589-421d-a390-ac7582d25a2a None admin] RBAC: Adding query filter params (role.id=8d259fab1ed64ced91f18487e90ee097, scope.project.id=bfe6c3d8da2743be9609184db2506303, user.id=6647eb1e8a1544ef9d3a9806f2d1fc51) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:51.941887 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be97163-d589-421d-a390-ac7582d25a2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:51.943309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be97163-d589-421d-a390-ac7582d25a2a None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:51.949420 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 558/1106] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:33:51 2022] GET /identity/v3/role_assignments?user.id=6647eb1e8a1544ef9d3a9806f2d1fc51&scope.project.id=bfe6c3d8da2743be9609184db2506303&role.id=8d259fab1ed64ced91f18487e90ee097 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:52.960981 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00468c89-c0bc-4999-8eb4-b108881fffa2 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:52.961870 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00468c89-c0bc-4999-8eb4-b108881fffa2 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:52.961870 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00468c89-c0bc-4999-8eb4-b108881fffa2 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:52.963114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 549/1107] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:52.968254 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ebdd3-82dc-4ae6-bad9-0f38fb3c2604 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:52.968658 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ebdd3-82dc-4ae6-bad9-0f38fb3c2604 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:52.968814 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ebdd3-82dc-4ae6-bad9-0f38fb3c2604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:52.986008 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-628ebdd3-82dc-4ae6-bad9-0f38fb3c2604 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:52.987170 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-628ebdd3-82dc-4ae6-bad9-0f38fb3c2604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:53.052649 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 559/1108] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:53.077988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00468c89-c0bc-4999-8eb4-b108881fffa2 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:53.176089 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00468c89-c0bc-4999-8eb4-b108881fffa2 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:53.208511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98603c15-1cf9-4f6a-9ee9-3c608ea0b87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:53.209316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98603c15-1cf9-4f6a-9ee9-3c608ea0b87e None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:53.209604 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98603c15-1cf9-4f6a-9ee9-3c608ea0b87e None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:53.209883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98603c15-1cf9-4f6a-9ee9-3c608ea0b87e None admin] PATH_INFO: `/v3/domains/default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:53.211097 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98603c15-1cf9-4f6a-9ee9-3c608ea0b87e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:53.212601 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98603c15-1cf9-4f6a-9ee9-3c608ea0b87e None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:53.214225 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 550/1109] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:33:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:53.219686 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ebdd3-82dc-4ae6-bad9-0f38fb3c2604 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:53.306936 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:53.315303 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ebdd3-82dc-4ae6-bad9-0f38fb3c2604 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:53.348467 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd726e18-c44e-48a8-8097-454fcf84965f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:53.349157 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd726e18-c44e-48a8-8097-454fcf84965f None admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:53.349439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd726e18-c44e-48a8-8097-454fcf84965f None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:53.349714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd726e18-c44e-48a8-8097-454fcf84965f None admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:53.350422 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd726e18-c44e-48a8-8097-454fcf84965f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:53.352483 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd726e18-c44e-48a8-8097-454fcf84965f None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:53.372062 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 560/1110] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jul 6 13:33:53 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:33:55.434461 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21f1f3-fbc0-4602-9502-11b142cc87a3 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:55.435019 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21f1f3-fbc0-4602-9502-11b142cc87a3 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:55.435019 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21f1f3-fbc0-4602-9502-11b142cc87a3 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:55.436362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 551/1111] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:55.441464 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b01e6eb-1fd9-450b-b4ec-e9874d6aa62d None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:55.441869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b01e6eb-1fd9-450b-b4ec-e9874d6aa62d None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:55.442028 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b01e6eb-1fd9-450b-b4ec-e9874d6aa62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:55.459023 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-8b01e6eb-1fd9-450b-b4ec-e9874d6aa62d None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:55.460200 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-8b01e6eb-1fd9-450b-b4ec-e9874d6aa62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:55.524650 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 561/1112] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:55.547932 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a21f1f3-fbc0-4602-9502-11b142cc87a3 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:55.566544 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:55.649708 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a21f1f3-fbc0-4602-9502-11b142cc87a3 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:55.683119 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9610eb-ffb4-472e-8dcf-265998e7456b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:55.683970 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9610eb-ffb4-472e-8dcf-265998e7456b None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:55.684257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9610eb-ffb4-472e-8dcf-265998e7456b None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:55.684534 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9610eb-ffb4-472e-8dcf-265998e7456b None admin] PATH_INFO: `/v3/roles/534400e7b28546a8a04c08f825c28566` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:55.685982 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9610eb-ffb4-472e-8dcf-265998e7456b None admin] RBAC: Authorizing `identity:get_role(role_id=534400e7b28546a8a04c08f825c28566)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:55.687336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9610eb-ffb4-472e-8dcf-265998e7456b None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:55.688692 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 552/1113] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:33:55 2022] GET /identity/v3/roles/534400e7b28546a8a04c08f825c28566 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:55.694501 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b01e6eb-1fd9-450b-b4ec-e9874d6aa62d None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:55.769198 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:55.791667 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b01e6eb-1fd9-450b-b4ec-e9874d6aa62d None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:55.823312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c197f9-a264-4a73-a20a-3e03c1a83a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:55.824116 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c197f9-a264-4a73-a20a-3e03c1a83a8d None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:55.824398 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c197f9-a264-4a73-a20a-3e03c1a83a8d None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:55.824671 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c197f9-a264-4a73-a20a-3e03c1a83a8d None admin] PATH_INFO: `/v3/users/e8bf6325b8994bd7988b68bccc9748d5` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:55.837836 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c197f9-a264-4a73-a20a-3e03c1a83a8d None admin] RBAC: Authorizing `identity:get_user(user_id=e8bf6325b8994bd7988b68bccc9748d5)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:55.839251 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c197f9-a264-4a73-a20a-3e03c1a83a8d None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:55.840868 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 562/1114] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:33:55 2022] GET /identity/v3/users/e8bf6325b8994bd7988b68bccc9748d5 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:55.846589 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9610eb-ffb4-472e-8dcf-265998e7456b None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:55.928834 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:55.945329 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9610eb-ffb4-472e-8dcf-265998e7456b None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:55.978426 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bfff7b-f387-4d01-b336-c1c0e586c0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:55.979061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfff7b-f387-4d01-b336-c1c0e586c0b4 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:55.979343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfff7b-f387-4d01-b336-c1c0e586c0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:55.979613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfff7b-f387-4d01-b336-c1c0e586c0b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:56.043047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bfff7b-f387-4d01-b336-c1c0e586c0b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:56.044467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bfff7b-f387-4d01-b336-c1c0e586c0b4 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:56.122351 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:56.138629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 553/1115] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:33:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 13:33:56.144487 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c197f9-a264-4a73-a20a-3e03c1a83a8d None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:56.239622 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c197f9-a264-4a73-a20a-3e03c1a83a8d None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:56.271715 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd99aa-7e20-4f27-8321-4636cd58b49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:56.272561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd99aa-7e20-4f27-8321-4636cd58b49e None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:56.272843 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd99aa-7e20-4f27-8321-4636cd58b49e None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:56.273148 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd99aa-7e20-4f27-8321-4636cd58b49e None admin] PATH_INFO: `/v3/projects/9fa3428bf0d3459a8e7e86d666e373e6` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:56.274690 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd99aa-7e20-4f27-8321-4636cd58b49e None admin] RBAC: Authorizing `identity:get_project(project_id=9fa3428bf0d3459a8e7e86d666e373e6)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:56.276079 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd99aa-7e20-4f27-8321-4636cd58b49e None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:56.277764 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 563/1116] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:33:56 2022] GET /identity/v3/projects/9fa3428bf0d3459a8e7e86d666e373e6 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:56.283723 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bfff7b-f387-4d01-b336-c1c0e586c0b4 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:56.379169 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bfff7b-f387-4d01-b336-c1c0e586c0b4 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:56.411408 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4cd7eb-784e-4ec1-ae41-b767d2058700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:56.412048 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4cd7eb-784e-4ec1-ae41-b767d2058700 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:56.412329 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4cd7eb-784e-4ec1-ae41-b767d2058700 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:56.412604 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4cd7eb-784e-4ec1-ae41-b767d2058700 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:56.413450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4cd7eb-784e-4ec1-ae41-b767d2058700 None admin] RBAC: Adding query filter params (role.id=534400e7b28546a8a04c08f825c28566, scope.project.id=9fa3428bf0d3459a8e7e86d666e373e6, user.id=e8bf6325b8994bd7988b68bccc9748d5) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:33:56.413763 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4cd7eb-784e-4ec1-ae41-b767d2058700 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:56.415424 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4cd7eb-784e-4ec1-ae41-b767d2058700 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:56.420624 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 554/1117] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:33:56 2022] GET /identity/v3/role_assignments?user.id=e8bf6325b8994bd7988b68bccc9748d5&scope.project.id=9fa3428bf0d3459a8e7e86d666e373e6&role.id=534400e7b28546a8a04c08f825c28566 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:58.410145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd51f8-8fa4-46ef-bd8c-c213341eb83d None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:58.410659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd51f8-8fa4-46ef-bd8c-c213341eb83d None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:58.410770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd51f8-8fa4-46ef-bd8c-c213341eb83d None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:58.412049 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 564/1118] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:33:58.417188 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c1ccb-8d28-487c-8ff2-c6131d099956 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:58.417617 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c1ccb-8d28-487c-8ff2-c6131d099956 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:58.417754 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c1ccb-8d28-487c-8ff2-c6131d099956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:58.434198 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-3a3c1ccb-8d28-487c-8ff2-c6131d099956 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:33:58.435349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-3a3c1ccb-8d28-487c-8ff2-c6131d099956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:33:58.477601 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:58.499798 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 555/1119] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:33:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:33:58.522246 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd51f8-8fa4-46ef-bd8c-c213341eb83d None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:58.526580 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:58.640172 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd51f8-8fa4-46ef-bd8c-c213341eb83d None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:58.672413 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415df3c6-f14f-4e21-a462-8332c4158554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:58.673287 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415df3c6-f14f-4e21-a462-8332c4158554 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:58.673822 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415df3c6-f14f-4e21-a462-8332c4158554 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:58.674099 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415df3c6-f14f-4e21-a462-8332c4158554 None admin] PATH_INFO: `/v3/roles/534400e7b28546a8a04c08f825c28566` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:58.675556 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415df3c6-f14f-4e21-a462-8332c4158554 None admin] RBAC: Authorizing `identity:get_role(role_id=534400e7b28546a8a04c08f825c28566)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:58.676904 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415df3c6-f14f-4e21-a462-8332c4158554 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:58.678296 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 565/1120] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:33:58 2022] GET /identity/v3/roles/534400e7b28546a8a04c08f825c28566 => generated 223 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:58.684603 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3c1ccb-8d28-487c-8ff2-c6131d099956 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:58.780668 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3c1ccb-8d28-487c-8ff2-c6131d099956 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:58.814141 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef9e40c-1795-46ce-8eae-51285ad11a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:58.814999 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9e40c-1795-46ce-8eae-51285ad11a37 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:58.815283 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9e40c-1795-46ce-8eae-51285ad11a37 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:58.815556 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9e40c-1795-46ce-8eae-51285ad11a37 None admin] PATH_INFO: `/v3/users/e8bf6325b8994bd7988b68bccc9748d5` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:58.816732 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef9e40c-1795-46ce-8eae-51285ad11a37 None admin] RBAC: Authorizing `identity:get_user(user_id=e8bf6325b8994bd7988b68bccc9748d5)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:58.818195 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef9e40c-1795-46ce-8eae-51285ad11a37 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:58.819817 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 556/1121] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:33:58 2022] GET /identity/v3/users/e8bf6325b8994bd7988b68bccc9748d5 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:58.826213 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415df3c6-f14f-4e21-a462-8332c4158554 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:58.893405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:58.926386 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415df3c6-f14f-4e21-a462-8332c4158554 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:58.959357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181b1df-7bef-4c84-9d72-80f5ae44e5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:58.960209 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181b1df-7bef-4c84-9d72-80f5ae44e5db None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:58.960490 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181b1df-7bef-4c84-9d72-80f5ae44e5db None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:58.960761 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181b1df-7bef-4c84-9d72-80f5ae44e5db None admin] PATH_INFO: `/v3/projects/9fa3428bf0d3459a8e7e86d666e373e6` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:58.962201 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b181b1df-7bef-4c84-9d72-80f5ae44e5db None admin] RBAC: Authorizing `identity:get_project(project_id=9fa3428bf0d3459a8e7e86d666e373e6)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:58.963616 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b181b1df-7bef-4c84-9d72-80f5ae44e5db None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:58.965359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 566/1122] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:33:58 2022] GET /identity/v3/projects/9fa3428bf0d3459a8e7e86d666e373e6 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:33:58.971334 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef9e40c-1795-46ce-8eae-51285ad11a37 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:33:59.017344 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:33:59.070275 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef9e40c-1795-46ce-8eae-51285ad11a37 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:33:59.102995 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1beaf2-359c-4d36-a64c-50d7057a86c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:33:59.103867 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1beaf2-359c-4d36-a64c-50d7057a86c5 None admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:33:59.104149 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1beaf2-359c-4d36-a64c-50d7057a86c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:33:59.104421 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1beaf2-359c-4d36-a64c-50d7057a86c5 None admin] PATH_INFO: `/v3/projects/9fa3428bf0d3459a8e7e86d666e373e6/users/e8bf6325b8994bd7988b68bccc9748d5/roles/534400e7b28546a8a04c08f825c28566` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:33:59.106610 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1beaf2-359c-4d36-a64c-50d7057a86c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=9fa3428bf0d3459a8e7e86d666e373e6, user_id=e8bf6325b8994bd7988b68bccc9748d5, role_id=534400e7b28546a8a04c08f825c28566)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:33:59.108021 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1beaf2-359c-4d36-a64c-50d7057a86c5 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:33:59.123016 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 557/1123] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jul 6 13:33:58 2022] PUT /identity/v3/projects/9fa3428bf0d3459a8e7e86d666e373e6/users/e8bf6325b8994bd7988b68bccc9748d5/roles/534400e7b28546a8a04c08f825c28566 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:34:01.108100 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961ac7d-948b-42ee-a590-6f383b3abcea None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:01.108658 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961ac7d-948b-42ee-a590-6f383b3abcea None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:01.108761 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961ac7d-948b-42ee-a590-6f383b3abcea None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:01.110089 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 567/1124] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:34:01.115216 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337c978-9be1-4d13-a5a5-0419a86968b3 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:01.115628 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337c978-9be1-4d13-a5a5-0419a86968b3 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:01.115777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337c978-9be1-4d13-a5a5-0419a86968b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:01.135197 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-2337c978-9be1-4d13-a5a5-0419a86968b3 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:34:01.136385 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-2337c978-9be1-4d13-a5a5-0419a86968b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:34:01.201775 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 558/1125] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:34:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:34:01.225160 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961ac7d-948b-42ee-a590-6f383b3abcea None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:01.324496 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961ac7d-948b-42ee-a590-6f383b3abcea None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:01.357194 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d88a22-019b-476b-8017-b4d6c039e16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:01.358037 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d88a22-019b-476b-8017-b4d6c039e16e None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:01.358370 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d88a22-019b-476b-8017-b4d6c039e16e None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:01.358598 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d88a22-019b-476b-8017-b4d6c039e16e None admin] PATH_INFO: `/v3/roles/534400e7b28546a8a04c08f825c28566` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:01.360042 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d88a22-019b-476b-8017-b4d6c039e16e None admin] RBAC: Authorizing `identity:get_role(role_id=534400e7b28546a8a04c08f825c28566)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:01.361425 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d88a22-019b-476b-8017-b4d6c039e16e None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:01.362788 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 568/1126] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:34:01 2022] GET /identity/v3/roles/534400e7b28546a8a04c08f825c28566 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:34:01.368691 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337c978-9be1-4d13-a5a5-0419a86968b3 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:01.401780 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:01.464048 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337c978-9be1-4d13-a5a5-0419a86968b3 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:01.497546 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55580832-9201-481e-b35d-aa8dacc93cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:01.498398 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55580832-9201-481e-b35d-aa8dacc93cd3 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:01.498675 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55580832-9201-481e-b35d-aa8dacc93cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:01.498945 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55580832-9201-481e-b35d-aa8dacc93cd3 None admin] PATH_INFO: `/v3/users/e8bf6325b8994bd7988b68bccc9748d5` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:01.500101 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55580832-9201-481e-b35d-aa8dacc93cd3 None admin] RBAC: Authorizing `identity:get_user(user_id=e8bf6325b8994bd7988b68bccc9748d5)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:01.501556 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55580832-9201-481e-b35d-aa8dacc93cd3 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:01.503178 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 559/1127] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:34:01 2022] GET /identity/v3/users/e8bf6325b8994bd7988b68bccc9748d5 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:34:01.509319 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d88a22-019b-476b-8017-b4d6c039e16e None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:01.519712 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:01.606050 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d88a22-019b-476b-8017-b4d6c039e16e None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:01.639086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d4972-a096-4be2-a240-f3e91ab0c85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:01.639668 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d4972-a096-4be2-a240-f3e91ab0c85a None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:01.639949 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d4972-a096-4be2-a240-f3e91ab0c85a None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:01.640227 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d4972-a096-4be2-a240-f3e91ab0c85a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:01.705272 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08d4972-a096-4be2-a240-f3e91ab0c85a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:01.706693 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08d4972-a096-4be2-a240-f3e91ab0c85a None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:01.716358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:01.800571 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 569/1128] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:34:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 13:34:01.807335 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55580832-9201-481e-b35d-aa8dacc93cd3 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:01.902677 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55580832-9201-481e-b35d-aa8dacc93cd3 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:01.904925 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:01.936811 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700482fe-925d-4d41-9cf0-abd042914b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:01.937712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700482fe-925d-4d41-9cf0-abd042914b7c None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:01.937992 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700482fe-925d-4d41-9cf0-abd042914b7c None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:01.938276 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700482fe-925d-4d41-9cf0-abd042914b7c None admin] PATH_INFO: `/v3/projects/9fa3428bf0d3459a8e7e86d666e373e6` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:01.939459 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700482fe-925d-4d41-9cf0-abd042914b7c None admin] RBAC: Authorizing `identity:get_project(project_id=9fa3428bf0d3459a8e7e86d666e373e6)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:01.940857 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700482fe-925d-4d41-9cf0-abd042914b7c None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:01.942561 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 560/1129] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:34:01 2022] GET /identity/v3/projects/9fa3428bf0d3459a8e7e86d666e373e6 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:34:01.948607 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d4972-a096-4be2-a240-f3e91ab0c85a None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:02.042614 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d4972-a096-4be2-a240-f3e91ab0c85a None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:02.052529 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:02.077400 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2fff9d-d01b-4e08-a943-cb192a26528b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:02.078039 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fff9d-d01b-4e08-a943-cb192a26528b None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:02.078322 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fff9d-d01b-4e08-a943-cb192a26528b None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:02.078602 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fff9d-d01b-4e08-a943-cb192a26528b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:02.079429 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2fff9d-d01b-4e08-a943-cb192a26528b None admin] RBAC: Adding query filter params (role.id=534400e7b28546a8a04c08f825c28566, scope.project.id=9fa3428bf0d3459a8e7e86d666e373e6, user.id=e8bf6325b8994bd7988b68bccc9748d5) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:34:02.079740 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2fff9d-d01b-4e08-a943-cb192a26528b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:02.081175 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2fff9d-d01b-4e08-a943-cb192a26528b None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:02.086760 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 570/1130] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:34:01 2022] GET /identity/v3/role_assignments?user.id=e8bf6325b8994bd7988b68bccc9748d5&scope.project.id=9fa3428bf0d3459a8e7e86d666e373e6&role.id=534400e7b28546a8a04c08f825c28566 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:34:04.097411 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ffad1-a889-4367-9888-23e655081182 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:04.097968 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ffad1-a889-4367-9888-23e655081182 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:04.097968 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ffad1-a889-4367-9888-23e655081182 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:04.099313 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 561/1131] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:34:04.104681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c13fc9-4185-4e7d-a276-cb3a11ce9520 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:04.105085 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c13fc9-4185-4e7d-a276-cb3a11ce9520 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:04.105262 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c13fc9-4185-4e7d-a276-cb3a11ce9520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:04.122103 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-59c13fc9-4185-4e7d-a276-cb3a11ce9520 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:34:04.123261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-59c13fc9-4185-4e7d-a276-cb3a11ce9520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:34:04.187538 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 571/1132] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:34:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:34:04.210219 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ffad1-a889-4367-9888-23e655081182 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:04.327481 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ffad1-a889-4367-9888-23e655081182 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:04.360853 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28120add-8ad2-479d-9716-55966f7a4542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:04.361613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28120add-8ad2-479d-9716-55966f7a4542 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:04.361901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28120add-8ad2-479d-9716-55966f7a4542 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:04.362179 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28120add-8ad2-479d-9716-55966f7a4542 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:04.363333 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28120add-8ad2-479d-9716-55966f7a4542 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:04.364761 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28120add-8ad2-479d-9716-55966f7a4542 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:04.366391 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 562/1133] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:34:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:34:04.372895 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c13fc9-4185-4e7d-a276-cb3a11ce9520 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:04.466567 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c13fc9-4185-4e7d-a276-cb3a11ce9520 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:04.468017 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:04.500051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb9670b-f988-41cf-b69f-e27fcf4f6b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:04.500700 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9670b-f988-41cf-b69f-e27fcf4f6b84 None admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:04.500985 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9670b-f988-41cf-b69f-e27fcf4f6b84 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:04.501294 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9670b-f988-41cf-b69f-e27fcf4f6b84 None admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:04.501999 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb9670b-f988-41cf-b69f-e27fcf4f6b84 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:04.503404 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb9670b-f988-41cf-b69f-e27fcf4f6b84 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:04.537393 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 572/1134] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Jul 6 13:34:04 2022] POST /identity/v3/projects => generated 311 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:34:06.586955 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e37265-24d3-4c21-a02f-ea212c795dc5 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:06.587474 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e37265-24d3-4c21-a02f-ea212c795dc5 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:06.587833 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e37265-24d3-4c21-a02f-ea212c795dc5 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:06.589704 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 563/1135] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:34:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:34:06.596524 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c396d36-36f8-464b-96c4-88fc853d8417 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:06.596974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c396d36-36f8-464b-96c4-88fc853d8417 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:06.597116 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c396d36-36f8-464b-96c4-88fc853d8417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:06.614240 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-1c396d36-36f8-464b-96c4-88fc853d8417 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:34:06.615404 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-1c396d36-36f8-464b-96c4-88fc853d8417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:34:06.678788 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 573/1136] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:34:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:34:06.703576 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e37265-24d3-4c21-a02f-ea212c795dc5 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:06.747098 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:06.808498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e37265-24d3-4c21-a02f-ea212c795dc5 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:06.841363 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609dd05-4d3e-4b8f-9f9f-b70a725df074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:06.842079 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609dd05-4d3e-4b8f-9f9f-b70a725df074 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:06.842365 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609dd05-4d3e-4b8f-9f9f-b70a725df074 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:06.842651 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609dd05-4d3e-4b8f-9f9f-b70a725df074 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:06.843813 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6609dd05-4d3e-4b8f-9f9f-b70a725df074 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:06.845260 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6609dd05-4d3e-4b8f-9f9f-b70a725df074 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:06.846858 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 564/1137] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:34:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:34:06.852774 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c396d36-36f8-464b-96c4-88fc853d8417 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:06.938951 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:06.956378 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c396d36-36f8-464b-96c4-88fc853d8417 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:06.988202 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f465319-6d67-42ca-b2c0-68f4dc2b9487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:06.988871 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f465319-6d67-42ca-b2c0-68f4dc2b9487 None admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:06.989188 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f465319-6d67-42ca-b2c0-68f4dc2b9487 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:06.989471 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f465319-6d67-42ca-b2c0-68f4dc2b9487 None admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:06.990181 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f465319-6d67-42ca-b2c0-68f4dc2b9487 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:06.991600 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f465319-6d67-42ca-b2c0-68f4dc2b9487 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:07.012548 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 574/1138] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jul 6 13:34:06 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:34:09.030889 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caad5b0-a12f-404a-b83d-1ee1b13f1be7 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:09.031450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caad5b0-a12f-404a-b83d-1ee1b13f1be7 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:09.031450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caad5b0-a12f-404a-b83d-1ee1b13f1be7 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:09.032788 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 565/1139] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:34:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:34:09.038001 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfe2a8-df61-4506-923d-50550057b201 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:09.038434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfe2a8-df61-4506-923d-50550057b201 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:09.038561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfe2a8-df61-4506-923d-50550057b201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:09.055172 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-6bdfe2a8-df61-4506-923d-50550057b201 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:34:09.056843 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-6bdfe2a8-df61-4506-923d-50550057b201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:34:09.120533 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 575/1140] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:34:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:34:09.144470 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caad5b0-a12f-404a-b83d-1ee1b13f1be7 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:09.247034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caad5b0-a12f-404a-b83d-1ee1b13f1be7 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:09.255841 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:09.281397 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f09d72-8684-45a5-89f7-fb0386f5fa9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:09.282115 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f09d72-8684-45a5-89f7-fb0386f5fa9d None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:09.282398 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f09d72-8684-45a5-89f7-fb0386f5fa9d None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:09.282675 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f09d72-8684-45a5-89f7-fb0386f5fa9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:09.293716 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f09d72-8684-45a5-89f7-fb0386f5fa9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:34:09.301578 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-54f09d72-8684-45a5-89f7-fb0386f5fa9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:34:09.301578 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:34:09.301578 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:34:09.301578 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:34:09.301578 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:34:09.301578 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:34:09.301578 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:34:09.301578 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:34:09.301578 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:34:09.301578 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:34:09.301578 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:34:09.301578 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:34:09.301578 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:34:09.301578 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:34:09.301578 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:34:09.301578 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:34:09.302778 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:34:09.302778 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:34:09.302778 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:34:09.302778 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:34:09.302778 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:34:09.302778 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:34:09.302778 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:34:09.302778 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:34:09.302778 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:34:09.302778 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:34:09.302778 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:34:09.302778 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:34:09.302778 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:34:09.302778 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:34:09.302778 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:34:09.302778 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:34:09.303886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:34:09.303886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:34:09.303886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:34:09.303886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:34:09.303886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:34:09.303886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:34:09.303886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:34:09.303886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:34:09.303886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:34:09.303886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:34:09.303886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:34:09.303886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:34:09.303886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 566/1141] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:34:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:34:09.309237 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfe2a8-df61-4506-923d-50550057b201 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:09.376654 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:09.407648 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfe2a8-df61-4506-923d-50550057b201 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:09.441232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bcd54f-c83f-411c-ab3c-8c38ba49268d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:09.441886 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bcd54f-c83f-411c-ab3c-8c38ba49268d None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:09.442172 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bcd54f-c83f-411c-ab3c-8c38ba49268d None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:09.442443 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bcd54f-c83f-411c-ab3c-8c38ba49268d None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:09.443181 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bcd54f-c83f-411c-ab3c-8c38ba49268d None admin] RBAC: Adding query filter params (name=admin) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:34:09.443484 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bcd54f-c83f-411c-ab3c-8c38ba49268d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:09.444841 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bcd54f-c83f-411c-ab3c-8c38ba49268d None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:09.453867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 576/1142] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:34:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:34:09.459792 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f09d72-8684-45a5-89f7-fb0386f5fa9d None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:09.555651 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f09d72-8684-45a5-89f7-fb0386f5fa9d None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:09.587050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462938bf-92fd-4fc2-9b0b-f441ec278fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:09.588749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462938bf-92fd-4fc2-9b0b-f441ec278fa1 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:09.589050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462938bf-92fd-4fc2-9b0b-f441ec278fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:09.589324 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462938bf-92fd-4fc2-9b0b-f441ec278fa1 None admin] PATH_INFO: `/v3/users/ebd9645cfe6746bcafddde5c7ec4cb86` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:09.601659 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462938bf-92fd-4fc2-9b0b-f441ec278fa1 None admin] RBAC: Authorizing `identity:get_user(user_id=ebd9645cfe6746bcafddde5c7ec4cb86)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:09.603182 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462938bf-92fd-4fc2-9b0b-f441ec278fa1 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:09.604917 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 567/1143] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:34:09 2022] GET /identity/v3/users/ebd9645cfe6746bcafddde5c7ec4cb86 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:34:09.610443 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bcd54f-c83f-411c-ab3c-8c38ba49268d None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:09.705178 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bcd54f-c83f-411c-ab3c-8c38ba49268d None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:09.732481 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:09.737611 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de96e04-6165-49c7-b951-6079e9518ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:09.738249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de96e04-6165-49c7-b951-6079e9518ad2 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:09.738528 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de96e04-6165-49c7-b951-6079e9518ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:09.738801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de96e04-6165-49c7-b951-6079e9518ad2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:09.802389 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de96e04-6165-49c7-b951-6079e9518ad2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:09.803810 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de96e04-6165-49c7-b951-6079e9518ad2 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:09.896483 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 577/1144] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:34:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 13:34:09.902836 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462938bf-92fd-4fc2-9b0b-f441ec278fa1 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:09.929905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:10.001416 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462938bf-92fd-4fc2-9b0b-f441ec278fa1 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:10.034310 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7f861-98b9-49f5-8cc0-926acf3443f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:10.035157 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7f861-98b9-49f5-8cc0-926acf3443f8 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:10.035437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7f861-98b9-49f5-8cc0-926acf3443f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:10.035711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7f861-98b9-49f5-8cc0-926acf3443f8 None admin] PATH_INFO: `/v3/projects/9cdf0fdca24d419d86ba4135927fa822` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:10.036860 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c7f861-98b9-49f5-8cc0-926acf3443f8 None admin] RBAC: Authorizing `identity:get_project(project_id=9cdf0fdca24d419d86ba4135927fa822)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:10.038305 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c7f861-98b9-49f5-8cc0-926acf3443f8 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:10.039970 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 568/1145] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:34:09 2022] GET /identity/v3/projects/9cdf0fdca24d419d86ba4135927fa822 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:34:10.046540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de96e04-6165-49c7-b951-6079e9518ad2 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:10.075824 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:10.140218 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de96e04-6165-49c7-b951-6079e9518ad2 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:10.172379 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b676170-b139-4fa7-b49e-b814ca876958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:10.172960 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b676170-b139-4fa7-b49e-b814ca876958 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:10.173528 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b676170-b139-4fa7-b49e-b814ca876958 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:10.173804 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b676170-b139-4fa7-b49e-b814ca876958 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:10.174622 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b676170-b139-4fa7-b49e-b814ca876958 None admin] RBAC: Adding query filter params (role.id=795daa2c78354fd48e4a741e1182cc98, scope.project.id=9cdf0fdca24d419d86ba4135927fa822, user.id=ebd9645cfe6746bcafddde5c7ec4cb86) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:34:10.174928 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b676170-b139-4fa7-b49e-b814ca876958 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:10.176319 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b676170-b139-4fa7-b49e-b814ca876958 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:10.181559 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 578/1146] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:34:10 2022] GET /identity/v3/role_assignments?user.id=ebd9645cfe6746bcafddde5c7ec4cb86&scope.project.id=9cdf0fdca24d419d86ba4135927fa822&role.id=795daa2c78354fd48e4a741e1182cc98 => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:34:12.222792 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd3e3e-4974-44da-9b5d-5299c3c3d37f None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:12.223323 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd3e3e-4974-44da-9b5d-5299c3c3d37f None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:12.223323 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd3e3e-4974-44da-9b5d-5299c3c3d37f None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:12.224689 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 569/1147] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:34:12.230537 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dee189-734d-42f2-aaa7-65f82a1bee21 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:12.230912 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dee189-734d-42f2-aaa7-65f82a1bee21 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:12.231097 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dee189-734d-42f2-aaa7-65f82a1bee21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:12.247541 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-41dee189-734d-42f2-aaa7-65f82a1bee21 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:34:12.248711 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-41dee189-734d-42f2-aaa7-65f82a1bee21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:34:12.315201 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 579/1148] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:34:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:34:12.338637 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cd3e3e-4974-44da-9b5d-5299c3c3d37f None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:12.428421 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:12.436430 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cd3e3e-4974-44da-9b5d-5299c3c3d37f None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:12.469549 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ccd197-ed99-4637-b1cf-2b3fe6ccccd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:12.470352 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ccd197-ed99-4637-b1cf-2b3fe6ccccd5 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:12.470659 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ccd197-ed99-4637-b1cf-2b3fe6ccccd5 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:12.470931 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ccd197-ed99-4637-b1cf-2b3fe6ccccd5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:12.480706 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ccd197-ed99-4637-b1cf-2b3fe6ccccd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:34:12.488204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-67ccd197-ed99-4637-b1cf-2b3fe6ccccd5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:34:12.488204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:34:12.488204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:34:12.488204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:34:12.488204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:34:12.488204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:34:12.488204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:34:12.488204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:34:12.488204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:34:12.488204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:34:12.488204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:34:12.488204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:34:12.488204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:34:12.488204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:34:12.488204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:34:12.488204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:34:12.489362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:34:12.489362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:34:12.489362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:34:12.489362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:34:12.489362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:34:12.489362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:34:12.489362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:34:12.489362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:34:12.489362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:34:12.489362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:34:12.489362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:34:12.489362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:34:12.489362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:34:12.489362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:34:12.489362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:34:12.489362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:34:12.490418 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:34:12.490418 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:34:12.490418 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:34:12.490418 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:34:12.490418 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:34:12.490418 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:34:12.490418 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:34:12.490418 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:34:12.490418 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:34:12.490418 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:34:12.490418 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:34:12.490418 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:34:12.490418 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 570/1149] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:34:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:34:12.495953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dee189-734d-42f2-aaa7-65f82a1bee21 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:12.522061 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:12.595223 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dee189-734d-42f2-aaa7-65f82a1bee21 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:12.631980 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac238f78-b218-4f77-be2f-12e08c1177f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:12.632646 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac238f78-b218-4f77-be2f-12e08c1177f0 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:12.632934 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac238f78-b218-4f77-be2f-12e08c1177f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:12.633246 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac238f78-b218-4f77-be2f-12e08c1177f0 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:12.633991 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac238f78-b218-4f77-be2f-12e08c1177f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:34:12.635268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac238f78-b218-4f77-be2f-12e08c1177f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:12.636620 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac238f78-b218-4f77-be2f-12e08c1177f0 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:12.644447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 580/1150] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:34:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:34:12.651211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ccd197-ed99-4637-b1cf-2b3fe6ccccd5 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:12.747594 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ccd197-ed99-4637-b1cf-2b3fe6ccccd5 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:12.779836 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd2a28-019a-42df-a736-2a1f6a9cb43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:12.780684 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd2a28-019a-42df-a736-2a1f6a9cb43d None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:12.780965 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd2a28-019a-42df-a736-2a1f6a9cb43d None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:12.781268 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd2a28-019a-42df-a736-2a1f6a9cb43d None admin] PATH_INFO: `/v3/users/ebd9645cfe6746bcafddde5c7ec4cb86` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:12.782422 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd2a28-019a-42df-a736-2a1f6a9cb43d None admin] RBAC: Authorizing `identity:get_user(user_id=ebd9645cfe6746bcafddde5c7ec4cb86)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:12.783827 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd2a28-019a-42df-a736-2a1f6a9cb43d None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:12.785516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 571/1151] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:34:12 2022] GET /identity/v3/users/ebd9645cfe6746bcafddde5c7ec4cb86 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:34:12.791493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac238f78-b218-4f77-be2f-12e08c1177f0 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:12.877623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:12.895087 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac238f78-b218-4f77-be2f-12e08c1177f0 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:12.928080 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c622ed0-1acb-4a34-8930-e31279d20433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:12.928927 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c622ed0-1acb-4a34-8930-e31279d20433 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:12.929234 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c622ed0-1acb-4a34-8930-e31279d20433 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:12.929505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c622ed0-1acb-4a34-8930-e31279d20433 None admin] PATH_INFO: `/v3/projects/9cdf0fdca24d419d86ba4135927fa822` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:12.930658 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c622ed0-1acb-4a34-8930-e31279d20433 None admin] RBAC: Authorizing `identity:get_project(project_id=9cdf0fdca24d419d86ba4135927fa822)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:12.932720 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c622ed0-1acb-4a34-8930-e31279d20433 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:12.934435 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 581/1152] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:34:12 2022] GET /identity/v3/projects/9cdf0fdca24d419d86ba4135927fa822 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:34:12.941819 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd2a28-019a-42df-a736-2a1f6a9cb43d None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:12.967527 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:13.040307 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd2a28-019a-42df-a736-2a1f6a9cb43d None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:13.073513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa869be-93af-4873-97c5-e333090543af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:13.074380 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa869be-93af-4873-97c5-e333090543af None admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:13.074662 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa869be-93af-4873-97c5-e333090543af None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:13.074937 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa869be-93af-4873-97c5-e333090543af None admin] PATH_INFO: `/v3/projects/9cdf0fdca24d419d86ba4135927fa822/users/ebd9645cfe6746bcafddde5c7ec4cb86/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:13.076842 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa869be-93af-4873-97c5-e333090543af None admin] RBAC: Authorizing `identity:create_grant(project_id=9cdf0fdca24d419d86ba4135927fa822, user_id=ebd9645cfe6746bcafddde5c7ec4cb86, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:13.078282 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa869be-93af-4873-97c5-e333090543af None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:13.090422 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 572/1153] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jul 6 13:34:12 2022] PUT /identity/v3/projects/9cdf0fdca24d419d86ba4135927fa822/users/ebd9645cfe6746bcafddde5c7ec4cb86/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:34:15.115489 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238c182-1578-405e-ba62-ff1dc40e15c9 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:15.116047 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238c182-1578-405e-ba62-ff1dc40e15c9 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:15.116047 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238c182-1578-405e-ba62-ff1dc40e15c9 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:15.117459 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 582/1154] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:34:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:34:15.123024 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6bf27-a56c-442b-86d9-63849e9eb9bc None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:15.123437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6bf27-a56c-442b-86d9-63849e9eb9bc None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:15.123589 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6bf27-a56c-442b-86d9-63849e9eb9bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:15.140989 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-c6e6bf27-a56c-442b-86d9-63849e9eb9bc None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:34:15.142185 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-c6e6bf27-a56c-442b-86d9-63849e9eb9bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:34:15.208098 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 573/1155] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:34:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:34:15.231168 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238c182-1578-405e-ba62-ff1dc40e15c9 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:15.335571 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238c182-1578-405e-ba62-ff1dc40e15c9 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:15.367709 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625dff6e-e6e0-4671-b2a9-488b805047e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:15.369071 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625dff6e-e6e0-4671-b2a9-488b805047e9 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:15.369489 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625dff6e-e6e0-4671-b2a9-488b805047e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:15.369689 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625dff6e-e6e0-4671-b2a9-488b805047e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:15.379744 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625dff6e-e6e0-4671-b2a9-488b805047e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:34:15.387290 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-625dff6e-e6e0-4671-b2a9-488b805047e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:34:15.387290 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:34:15.387290 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:34:15.387290 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:34:15.387290 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:34:15.387290 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:34:15.387290 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:34:15.387290 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:34:15.387290 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:34:15.387290 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:34:15.387290 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:34:15.387290 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:34:15.387290 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:34:15.387290 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:34:15.387290 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:34:15.387290 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:34:15.388380 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:34:15.388380 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:34:15.388380 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:34:15.388380 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:34:15.388380 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:34:15.388380 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:34:15.388380 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:34:15.388380 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:34:15.388380 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:34:15.388380 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:34:15.388380 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:34:15.388380 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:34:15.388380 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:34:15.388380 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:34:15.388380 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:34:15.388380 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:34:15.389437 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:34:15.389437 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:34:15.389437 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:34:15.389437 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:34:15.389437 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:34:15.389437 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:34:15.389437 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:34:15.389437 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:34:15.389437 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:34:15.389437 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:34:15.389437 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:34:15.389437 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:34:15.389437 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 583/1156] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:34:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:34:15.394473 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6bf27-a56c-442b-86d9-63849e9eb9bc None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:15.408267 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:15.491809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6bf27-a56c-442b-86d9-63849e9eb9bc None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:15.524331 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5337928-897c-411d-8668-2d034b677920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:15.525048 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5337928-897c-411d-8668-2d034b677920 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:15.525331 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5337928-897c-411d-8668-2d034b677920 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:15.525609 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5337928-897c-411d-8668-2d034b677920 None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:15.526352 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5337928-897c-411d-8668-2d034b677920 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:34:15.526664 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5337928-897c-411d-8668-2d034b677920 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:15.528035 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5337928-897c-411d-8668-2d034b677920 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:15.535808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 574/1157] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:34:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:34:15.542010 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625dff6e-e6e0-4671-b2a9-488b805047e9 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:15.560220 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:15.638443 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625dff6e-e6e0-4671-b2a9-488b805047e9 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:15.670733 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bf534-a325-449e-928a-35c06683d773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:15.671580 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bf534-a325-449e-928a-35c06683d773 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:15.671865 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bf534-a325-449e-928a-35c06683d773 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:15.672142 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bf534-a325-449e-928a-35c06683d773 None admin] PATH_INFO: `/v3/users/ebd9645cfe6746bcafddde5c7ec4cb86` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:15.673589 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437bf534-a325-449e-928a-35c06683d773 None admin] RBAC: Authorizing `identity:get_user(user_id=ebd9645cfe6746bcafddde5c7ec4cb86)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:15.674993 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437bf534-a325-449e-928a-35c06683d773 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:15.676589 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 584/1158] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:34:15 2022] GET /identity/v3/users/ebd9645cfe6746bcafddde5c7ec4cb86 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:34:15.682356 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5337928-897c-411d-8668-2d034b677920 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:15.761704 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:15.785377 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5337928-897c-411d-8668-2d034b677920 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:15.817844 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c51c1c-e064-44c1-8dfc-a1050a776823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:15.818482 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c51c1c-e064-44c1-8dfc-a1050a776823 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:15.818767 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c51c1c-e064-44c1-8dfc-a1050a776823 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:15.819051 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c51c1c-e064-44c1-8dfc-a1050a776823 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:15.883176 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c51c1c-e064-44c1-8dfc-a1050a776823 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:15.884684 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c51c1c-e064-44c1-8dfc-a1050a776823 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:15.954055 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:15.979509 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 575/1159] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:34:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 06 13:34:15.985389 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bf534-a325-449e-928a-35c06683d773 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:16.065740 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:16.081219 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bf534-a325-449e-928a-35c06683d773 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:16.114097 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3280552-f229-421f-ae9a-c577a2d6c14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:16.114941 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3280552-f229-421f-ae9a-c577a2d6c14f None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:16.115226 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3280552-f229-421f-ae9a-c577a2d6c14f None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:16.115502 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3280552-f229-421f-ae9a-c577a2d6c14f None admin] PATH_INFO: `/v3/projects/9cdf0fdca24d419d86ba4135927fa822` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:16.116660 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3280552-f229-421f-ae9a-c577a2d6c14f None admin] RBAC: Authorizing `identity:get_project(project_id=9cdf0fdca24d419d86ba4135927fa822)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:16.118111 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3280552-f229-421f-ae9a-c577a2d6c14f None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:16.120056 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 585/1160] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:34:15 2022] GET /identity/v3/projects/9cdf0fdca24d419d86ba4135927fa822 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:34:16.126212 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c51c1c-e064-44c1-8dfc-a1050a776823 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:16.221991 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c51c1c-e064-44c1-8dfc-a1050a776823 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:16.254143 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea501a4-13d8-4bf9-a2e9-e2e1d17c6106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:16.254784 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea501a4-13d8-4bf9-a2e9-e2e1d17c6106 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:16.255069 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea501a4-13d8-4bf9-a2e9-e2e1d17c6106 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:16.255345 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea501a4-13d8-4bf9-a2e9-e2e1d17c6106 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:16.256170 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea501a4-13d8-4bf9-a2e9-e2e1d17c6106 None admin] RBAC: Adding query filter params (role.id=795daa2c78354fd48e4a741e1182cc98, scope.project.id=9cdf0fdca24d419d86ba4135927fa822, user.id=ebd9645cfe6746bcafddde5c7ec4cb86) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:34:16.256485 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea501a4-13d8-4bf9-a2e9-e2e1d17c6106 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:16.257928 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea501a4-13d8-4bf9-a2e9-e2e1d17c6106 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:16.264343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 576/1161] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:34:16 2022] GET /identity/v3/role_assignments?user.id=ebd9645cfe6746bcafddde5c7ec4cb86&scope.project.id=9cdf0fdca24d419d86ba4135927fa822&role.id=795daa2c78354fd48e4a741e1182cc98 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:34:18.317037 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb30e4-02c0-4d25-8a16-3c8ca31c93bc None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:18.317580 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb30e4-02c0-4d25-8a16-3c8ca31c93bc None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:18.317580 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb30e4-02c0-4d25-8a16-3c8ca31c93bc None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:18.318976 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 586/1162] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:34:18.325242 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d7b63-52fb-4664-add1-4d4273c021ad None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:18.325646 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d7b63-52fb-4664-add1-4d4273c021ad None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:18.325804 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d7b63-52fb-4664-add1-4d4273c021ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:18.342104 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-e62d7b63-52fb-4664-add1-4d4273c021ad None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:34:18.343281 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-e62d7b63-52fb-4664-add1-4d4273c021ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:34:18.372589 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:18.408885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 577/1163] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:34:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:34:18.432600 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb30e4-02c0-4d25-8a16-3c8ca31c93bc None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:18.528056 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb30e4-02c0-4d25-8a16-3c8ca31c93bc None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:18.560248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc99a497-5a92-435c-8ab7-d9204e018c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:18.560817 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99a497-5a92-435c-8ab7-d9204e018c9b None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:18.561124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99a497-5a92-435c-8ab7-d9204e018c9b None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:18.561399 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99a497-5a92-435c-8ab7-d9204e018c9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:18.581627 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:18.625959 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc99a497-5a92-435c-8ab7-d9204e018c9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:18.627376 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc99a497-5a92-435c-8ab7-d9204e018c9b None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:18.720468 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 587/1164] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:34:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 13:34:18.726374 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d7b63-52fb-4664-add1-4d4273c021ad None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:18.821611 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d7b63-52fb-4664-add1-4d4273c021ad None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:18.854531 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1b56d4-c98f-4354-ab41-e444e36e2ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:18.855317 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b56d4-c98f-4354-ab41-e444e36e2ba1 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:18.855599 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b56d4-c98f-4354-ab41-e444e36e2ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:18.855872 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b56d4-c98f-4354-ab41-e444e36e2ba1 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:18.862298 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1b56d4-c98f-4354-ab41-e444e36e2ba1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:18.863727 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1b56d4-c98f-4354-ab41-e444e36e2ba1 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:18.877529 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-0b1b56d4-c98f-4354-ab41-e444e36e2ba1 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 06 13:34:18.877529 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:34:18.877529 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 06 13:34:18.877529 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 06 13:34:18.877529 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:34:18.877529 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:34:18.877529 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:34:18.877529 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:34:18.877529 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 06 13:34:18.877529 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:34:18.877529 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 13:34:18.877529 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:34:18.877529 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:34:18.877529 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:34:18.877529 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:34:18.877529 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:34:18.877529 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:34:18.878764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:34:18.878764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:34:18.878764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:34:18.878764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:34:18.878764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:34:18.878764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:34:18.878764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:34:18.878764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:34:18.878764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:34:18.878764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:34:18.878764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:34:18.878764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:34:18.878764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:34:18.878764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:34:18.878764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:34:18.878764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:34:18.879809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:34:18.879809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:34:18.879809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:34:18.879809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:34:18.879809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:34:18.879809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:34:18.879809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:34:18.879809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:34:18.879809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:34:18.879809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:34:18.879809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 06 13:34:18.879809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:34:18.879809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 06 13:34:18.879809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:34:18.879809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 578/1165] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Jul 6 13:34:18 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:34:18.884663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc99a497-5a92-435c-8ab7-d9204e018c9b None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:18.935503 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:18.980639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc99a497-5a92-435c-8ab7-d9204e018c9b None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:19.013383 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e494634-56db-4305-b755-a0cbbe5b6e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:19.014030 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e494634-56db-4305-b755-a0cbbe5b6e05 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:19.014313 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e494634-56db-4305-b755-a0cbbe5b6e05 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:19.014590 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e494634-56db-4305-b755-a0cbbe5b6e05 None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:19.015309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e494634-56db-4305-b755-a0cbbe5b6e05 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:34:19.015614 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e494634-56db-4305-b755-a0cbbe5b6e05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:19.016962 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e494634-56db-4305-b755-a0cbbe5b6e05 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:19.023826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 588/1166] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Jul 6 13:34:18 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:34:21.052028 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d274b-ddb9-4bda-a9d8-763647501aa8 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:21.052553 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d274b-ddb9-4bda-a9d8-763647501aa8 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:21.052671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d274b-ddb9-4bda-a9d8-763647501aa8 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:21.053994 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 579/1167] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:34:21.059753 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de9da6-c95f-47ee-8142-e06e16d10015 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:21.060178 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de9da6-c95f-47ee-8142-e06e16d10015 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:21.060316 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de9da6-c95f-47ee-8142-e06e16d10015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:21.077596 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-b7de9da6-c95f-47ee-8142-e06e16d10015 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:34:21.078773 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-b7de9da6-c95f-47ee-8142-e06e16d10015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:34:21.144250 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 589/1168] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:34:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:34:21.167039 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d274b-ddb9-4bda-a9d8-763647501aa8 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:21.196075 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:21.268588 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d274b-ddb9-4bda-a9d8-763647501aa8 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:21.302439 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bbde9c-6b86-455e-a12b-510cb9c8ed78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:21.303045 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbde9c-6b86-455e-a12b-510cb9c8ed78 None admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:21.303333 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbde9c-6b86-455e-a12b-510cb9c8ed78 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:21.303610 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbde9c-6b86-455e-a12b-510cb9c8ed78 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:21.304351 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bbde9c-6b86-455e-a12b-510cb9c8ed78 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:21.306288 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bbde9c-6b86-455e-a12b-510cb9c8ed78 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:21.320512 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 580/1169] 10.222.0.20 () {60 vars in 1138 bytes} [Wed Jul 6 13:34:21 2022] POST /identity/v3/domains => generated 267 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:34:23.354156 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3154a6-d569-43d2-abe8-323cdfd865a9 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:23.354684 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3154a6-d569-43d2-abe8-323cdfd865a9 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:23.354779 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3154a6-d569-43d2-abe8-323cdfd865a9 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:23.356084 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 590/1170] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:34:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:34:23.361249 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0e526-f5a9-48fb-ab5d-583baa07b0bc None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:23.361661 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0e526-f5a9-48fb-ab5d-583baa07b0bc None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:23.361811 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0e526-f5a9-48fb-ab5d-583baa07b0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:23.378103 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-44c0e526-f5a9-48fb-ab5d-583baa07b0bc None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:34:23.379279 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-44c0e526-f5a9-48fb-ab5d-583baa07b0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:34:23.443445 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 581/1171] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:34:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:34:23.466497 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3154a6-d569-43d2-abe8-323cdfd865a9 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:23.512286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:23.564344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3154a6-d569-43d2-abe8-323cdfd865a9 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:23.596443 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400a8c91-9db9-416f-b61c-ee20ab826954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:23.597320 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a8c91-9db9-416f-b61c-ee20ab826954 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:23.597605 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a8c91-9db9-416f-b61c-ee20ab826954 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:23.597882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a8c91-9db9-416f-b61c-ee20ab826954 None admin] PATH_INFO: `/v3/domains/f4b442534c0e47baa66f26b651991c7d` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:23.608540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400a8c91-9db9-416f-b61c-ee20ab826954 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f4b442534c0e47baa66f26b651991c7d)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:23.609988 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400a8c91-9db9-416f-b61c-ee20ab826954 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:23.611588 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 591/1172] 10.222.0.20 () {56 vars in 1183 bytes} [Wed Jul 6 13:34:23 2022] GET /identity/v3/domains/f4b442534c0e47baa66f26b651991c7d => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:34:23.617283 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c0e526-f5a9-48fb-ab5d-583baa07b0bc None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:23.635464 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:23.713461 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c0e526-f5a9-48fb-ab5d-583baa07b0bc None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:23.747967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46082e1-9bc4-4822-bc72-202a82afd63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:23.748709 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46082e1-9bc4-4822-bc72-202a82afd63a None admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:23.748990 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46082e1-9bc4-4822-bc72-202a82afd63a None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:23.749296 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46082e1-9bc4-4822-bc72-202a82afd63a None admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:23.750004 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46082e1-9bc4-4822-bc72-202a82afd63a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:23.751433 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46082e1-9bc4-4822-bc72-202a82afd63a None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:23.777300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 582/1173] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Jul 6 13:34:23 2022] POST /identity/v3/projects => generated 361 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:34:25.815218 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd326d-3045-44b3-9dab-8e06049f7fd8 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:25.815791 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd326d-3045-44b3-9dab-8e06049f7fd8 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:25.815791 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd326d-3045-44b3-9dab-8e06049f7fd8 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:25.817174 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 592/1174] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:34:25.822328 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499ac90-233b-47ea-90d8-8ff33b39b3ec None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:25.822755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499ac90-233b-47ea-90d8-8ff33b39b3ec None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:25.822892 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499ac90-233b-47ea-90d8-8ff33b39b3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:25.840415 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-f499ac90-233b-47ea-90d8-8ff33b39b3ec None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:34:25.841625 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-f499ac90-233b-47ea-90d8-8ff33b39b3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:34:25.903410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:25.908495 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 583/1175] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:34:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:34:25.932573 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd326d-3045-44b3-9dab-8e06049f7fd8 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:26.030606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd326d-3045-44b3-9dab-8e06049f7fd8 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:26.037142 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:26.063716 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446e4379-a6d1-47ab-9321-ae7b462394e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:26.064559 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e4379-a6d1-47ab-9321-ae7b462394e0 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:26.064842 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e4379-a6d1-47ab-9321-ae7b462394e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:26.065156 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e4379-a6d1-47ab-9321-ae7b462394e0 None admin] PATH_INFO: `/v3/domains/f4b442534c0e47baa66f26b651991c7d` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:26.066369 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446e4379-a6d1-47ab-9321-ae7b462394e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f4b442534c0e47baa66f26b651991c7d)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:26.067733 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446e4379-a6d1-47ab-9321-ae7b462394e0 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:26.069372 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 593/1176] 10.222.0.20 () {56 vars in 1183 bytes} [Wed Jul 6 13:34:25 2022] GET /identity/v3/domains/f4b442534c0e47baa66f26b651991c7d => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:34:26.075628 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f499ac90-233b-47ea-90d8-8ff33b39b3ec None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:26.169624 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f499ac90-233b-47ea-90d8-8ff33b39b3ec None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:26.204080 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7fbf5-abbd-4a13-9efc-e32a83a626ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:26.204742 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7fbf5-abbd-4a13-9efc-e32a83a626ce None admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:26.205044 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7fbf5-abbd-4a13-9efc-e32a83a626ce None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:26.205325 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7fbf5-abbd-4a13-9efc-e32a83a626ce None admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:26.206032 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec7fbf5-abbd-4a13-9efc-e32a83a626ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:26.207477 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec7fbf5-abbd-4a13-9efc-e32a83a626ce None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:26.227258 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 584/1177] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jul 6 13:34:26 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:34:28.283178 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0a60a-cfa4-43ee-947e-46a2b828c411 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:28.283736 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0a60a-cfa4-43ee-947e-46a2b828c411 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:28.283736 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0a60a-cfa4-43ee-947e-46a2b828c411 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:28.285111 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 594/1178] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:34:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:34:28.291168 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9e37b-0afd-40e3-a33c-6489f5bda1aa None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:28.291583 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9e37b-0afd-40e3-a33c-6489f5bda1aa None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:28.291722 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9e37b-0afd-40e3-a33c-6489f5bda1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:28.309322 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-fdf9e37b-0afd-40e3-a33c-6489f5bda1aa None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:34:28.310499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-fdf9e37b-0afd-40e3-a33c-6489f5bda1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:34:28.358790 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:28.376511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 585/1179] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:34:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:34:28.399752 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec0a60a-cfa4-43ee-947e-46a2b828c411 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:28.498183 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec0a60a-cfa4-43ee-947e-46a2b828c411 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:28.531145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def4df8-878c-484d-9c71-bd085437c4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:28.531919 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def4df8-878c-484d-9c71-bd085437c4df None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:28.532204 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def4df8-878c-484d-9c71-bd085437c4df None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:28.532481 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def4df8-878c-484d-9c71-bd085437c4df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:28.542923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def4df8-878c-484d-9c71-bd085437c4df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:34:28.550609 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-3def4df8-878c-484d-9c71-bd085437c4df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:34:28.550609 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:34:28.550609 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:34:28.550609 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:34:28.550609 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:34:28.550609 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:34:28.550609 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:34:28.550609 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:34:28.550609 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:34:28.550609 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:34:28.550609 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:34:28.550609 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:34:28.550609 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:34:28.550609 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:34:28.550609 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:34:28.550609 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:34:28.551770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:34:28.551770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:34:28.551770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:34:28.551770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:34:28.551770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:34:28.551770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:34:28.551770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:34:28.551770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:34:28.551770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:34:28.551770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:34:28.551770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:34:28.551770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:34:28.551770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:34:28.551770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:34:28.551770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:34:28.551770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:34:28.552719 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:34:28.552719 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:34:28.552719 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:34:28.552719 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:34:28.552719 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:34:28.552719 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:34:28.552719 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:34:28.552719 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:34:28.552719 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:34:28.552719 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:34:28.552719 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:34:28.552719 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:34:28.552719 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 595/1180] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:34:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:34:28.557915 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9e37b-0afd-40e3-a33c-6489f5bda1aa None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:28.654483 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9e37b-0afd-40e3-a33c-6489f5bda1aa None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:28.690381 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa3563-d8cb-4bd4-adc0-a49efaa93a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:28.691041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa3563-d8cb-4bd4-adc0-a49efaa93a68 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:28.691323 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa3563-d8cb-4bd4-adc0-a49efaa93a68 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:28.691597 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa3563-d8cb-4bd4-adc0-a49efaa93a68 None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:28.692335 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa3563-d8cb-4bd4-adc0-a49efaa93a68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:34:28.692641 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa3563-d8cb-4bd4-adc0-a49efaa93a68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:28.694032 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa3563-d8cb-4bd4-adc0-a49efaa93a68 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:28.701797 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 586/1181] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:34:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:34:28.707581 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def4df8-878c-484d-9c71-bd085437c4df None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:28.741398 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:28.804718 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def4df8-878c-484d-9c71-bd085437c4df None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:28.836269 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3306d-022a-4d32-ad3e-ff15b7ef59a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:28.837674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3306d-022a-4d32-ad3e-ff15b7ef59a3 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:28.837950 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3306d-022a-4d32-ad3e-ff15b7ef59a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:28.838220 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3306d-022a-4d32-ad3e-ff15b7ef59a3 None admin] PATH_INFO: `/v3/users/91d2f93c10dc4160bfae808333c3edf6` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:28.850674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3306d-022a-4d32-ad3e-ff15b7ef59a3 None admin] RBAC: Authorizing `identity:get_user(user_id=91d2f93c10dc4160bfae808333c3edf6)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:28.852694 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3306d-022a-4d32-ad3e-ff15b7ef59a3 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:28.854461 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 596/1182] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:34:28 2022] GET /identity/v3/users/91d2f93c10dc4160bfae808333c3edf6 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:34:28.860082 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa3563-d8cb-4bd4-adc0-a49efaa93a68 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:28.907486 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:28.955736 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa3563-d8cb-4bd4-adc0-a49efaa93a68 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:28.988104 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248d6ad-2544-4c1c-bccc-49ae50e41af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:28.988739 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248d6ad-2544-4c1c-bccc-49ae50e41af9 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:28.989045 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248d6ad-2544-4c1c-bccc-49ae50e41af9 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:28.989327 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248d6ad-2544-4c1c-bccc-49ae50e41af9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:29.053836 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d248d6ad-2544-4c1c-bccc-49ae50e41af9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:29.055742 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d248d6ad-2544-4c1c-bccc-49ae50e41af9 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:29.100710 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:29.150340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 587/1183] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:34:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 13:34:29.156124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3306d-022a-4d32-ad3e-ff15b7ef59a3 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:29.244944 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:29.253822 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3306d-022a-4d32-ad3e-ff15b7ef59a3 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:29.288398 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19b8733-b532-4dc2-bca1-50ae2c83e74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:29.289285 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b8733-b532-4dc2-bca1-50ae2c83e74b None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:29.289609 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b8733-b532-4dc2-bca1-50ae2c83e74b None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:29.290366 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b8733-b532-4dc2-bca1-50ae2c83e74b None admin] PATH_INFO: `/v3/projects/405ed24ac9c84d4da611d2f05b1efe42` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:29.291566 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19b8733-b532-4dc2-bca1-50ae2c83e74b None admin] RBAC: Authorizing `identity:get_project(project_id=405ed24ac9c84d4da611d2f05b1efe42)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:29.293013 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19b8733-b532-4dc2-bca1-50ae2c83e74b None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:29.294729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 597/1184] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:34:29 2022] GET /identity/v3/projects/405ed24ac9c84d4da611d2f05b1efe42 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:34:29.300961 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248d6ad-2544-4c1c-bccc-49ae50e41af9 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:29.398320 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248d6ad-2544-4c1c-bccc-49ae50e41af9 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:29.431404 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512c5cf-3eb2-41c0-9874-dc8c8ff68bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:29.432745 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512c5cf-3eb2-41c0-9874-dc8c8ff68bac None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:29.433064 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512c5cf-3eb2-41c0-9874-dc8c8ff68bac None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:29.433332 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512c5cf-3eb2-41c0-9874-dc8c8ff68bac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:29.434162 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e512c5cf-3eb2-41c0-9874-dc8c8ff68bac None admin] RBAC: Adding query filter params (role.id=795daa2c78354fd48e4a741e1182cc98, scope.project.id=405ed24ac9c84d4da611d2f05b1efe42, user.id=91d2f93c10dc4160bfae808333c3edf6) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:34:29.434476 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e512c5cf-3eb2-41c0-9874-dc8c8ff68bac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:29.435909 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e512c5cf-3eb2-41c0-9874-dc8c8ff68bac None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:29.441424 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 588/1185] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:34:29 2022] GET /identity/v3/role_assignments?user.id=91d2f93c10dc4160bfae808333c3edf6&scope.project.id=405ed24ac9c84d4da611d2f05b1efe42&role.id=795daa2c78354fd48e4a741e1182cc98 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:34:31.442879 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fabffc-c592-47a2-9898-eb774edc8127 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:31.443421 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fabffc-c592-47a2-9898-eb774edc8127 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:31.443421 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fabffc-c592-47a2-9898-eb774edc8127 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:31.444844 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 598/1186] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:34:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:34:31.450120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ab847-8a4c-42f6-a56b-a54a69722b25 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:31.450510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ab847-8a4c-42f6-a56b-a54a69722b25 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:31.450683 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ab847-8a4c-42f6-a56b-a54a69722b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:31.467290 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-1d7ab847-8a4c-42f6-a56b-a54a69722b25 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:34:31.468450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-1d7ab847-8a4c-42f6-a56b-a54a69722b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:34:31.475609 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:31.533460 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 589/1187] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:34:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:34:31.556018 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fabffc-c592-47a2-9898-eb774edc8127 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:31.654588 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fabffc-c592-47a2-9898-eb774edc8127 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:31.687279 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5b009b-df8d-4465-b3fd-1cee83ab5d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:31.688051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b009b-df8d-4465-b3fd-1cee83ab5d66 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:31.688335 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b009b-df8d-4465-b3fd-1cee83ab5d66 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:31.688608 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b009b-df8d-4465-b3fd-1cee83ab5d66 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:31.698387 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5b009b-df8d-4465-b3fd-1cee83ab5d66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:34:31.706119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-bb5b009b-df8d-4465-b3fd-1cee83ab5d66 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:34:31.706119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:34:31.706119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:34:31.706119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:34:31.706119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:34:31.706119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:34:31.706119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:34:31.706119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:34:31.706119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:34:31.706119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:34:31.706119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:34:31.706119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:34:31.706119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:34:31.706119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:34:31.706119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:34:31.706119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:34:31.707185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:34:31.707185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:34:31.707185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:34:31.707185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:34:31.707185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:34:31.707185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:34:31.707185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:34:31.707185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:34:31.707185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:34:31.707185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:34:31.707185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:34:31.707185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:34:31.707185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:34:31.707185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:34:31.707185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:34:31.707185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:34:31.708101 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:34:31.708101 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:34:31.708101 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:34:31.708101 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:34:31.708101 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:34:31.708101 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:34:31.708101 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:34:31.708101 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:34:31.708101 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:34:31.708101 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:34:31.708101 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:34:31.708101 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:34:31.708101 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 599/1188] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:34:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:34:31.713259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ab847-8a4c-42f6-a56b-a54a69722b25 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:31.813732 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ab847-8a4c-42f6-a56b-a54a69722b25 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:31.849740 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889a82f-aa4d-4636-aaaf-ecff66249d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:31.850407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889a82f-aa4d-4636-aaaf-ecff66249d46 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:31.852292 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889a82f-aa4d-4636-aaaf-ecff66249d46 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:31.852292 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889a82f-aa4d-4636-aaaf-ecff66249d46 None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:31.852292 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b889a82f-aa4d-4636-aaaf-ecff66249d46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:34:31.852292 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b889a82f-aa4d-4636-aaaf-ecff66249d46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:31.854937 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b889a82f-aa4d-4636-aaaf-ecff66249d46 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:31.862672 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 590/1189] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:34:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:34:31.868455 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5b009b-df8d-4465-b3fd-1cee83ab5d66 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:31.892672 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:31.972903 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5b009b-df8d-4465-b3fd-1cee83ab5d66 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:32.006065 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a95b2-a7a9-48a3-8c8a-e989ea8cf519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:32.006932 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a95b2-a7a9-48a3-8c8a-e989ea8cf519 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:32.007219 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a95b2-a7a9-48a3-8c8a-e989ea8cf519 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:32.007507 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a95b2-a7a9-48a3-8c8a-e989ea8cf519 None admin] PATH_INFO: `/v3/users/91d2f93c10dc4160bfae808333c3edf6` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:32.008703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a95b2-a7a9-48a3-8c8a-e989ea8cf519 None admin] RBAC: Authorizing `identity:get_user(user_id=91d2f93c10dc4160bfae808333c3edf6)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:32.010176 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a95b2-a7a9-48a3-8c8a-e989ea8cf519 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:32.012505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 600/1190] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:34:31 2022] GET /identity/v3/users/91d2f93c10dc4160bfae808333c3edf6 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:34:32.018340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889a82f-aa4d-4636-aaaf-ecff66249d46 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:32.022113 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:32.117196 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889a82f-aa4d-4636-aaaf-ecff66249d46 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:32.151801 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486bf02-0308-4886-8371-166047f4cbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:32.152681 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486bf02-0308-4886-8371-166047f4cbea None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:32.152968 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486bf02-0308-4886-8371-166047f4cbea None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:32.153278 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486bf02-0308-4886-8371-166047f4cbea None admin] PATH_INFO: `/v3/projects/405ed24ac9c84d4da611d2f05b1efe42` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:32.154455 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0486bf02-0308-4886-8371-166047f4cbea None admin] RBAC: Authorizing `identity:get_project(project_id=405ed24ac9c84d4da611d2f05b1efe42)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:32.155883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0486bf02-0308-4886-8371-166047f4cbea None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:32.157619 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 591/1191] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:34:32 2022] GET /identity/v3/projects/405ed24ac9c84d4da611d2f05b1efe42 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:34:32.163495 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a95b2-a7a9-48a3-8c8a-e989ea8cf519 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:32.249188 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:32.260949 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a95b2-a7a9-48a3-8c8a-e989ea8cf519 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:32.294489 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5d813-0baf-47dc-8271-0b3abc9fcb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:32.295385 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5d813-0baf-47dc-8271-0b3abc9fcb0c None admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:32.295670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5d813-0baf-47dc-8271-0b3abc9fcb0c None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:32.295947 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5d813-0baf-47dc-8271-0b3abc9fcb0c None admin] PATH_INFO: `/v3/projects/405ed24ac9c84d4da611d2f05b1efe42/users/91d2f93c10dc4160bfae808333c3edf6/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:32.297945 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d5d813-0baf-47dc-8271-0b3abc9fcb0c None admin] RBAC: Authorizing `identity:create_grant(project_id=405ed24ac9c84d4da611d2f05b1efe42, user_id=91d2f93c10dc4160bfae808333c3edf6, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:32.299636 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d5d813-0baf-47dc-8271-0b3abc9fcb0c None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:32.313137 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 601/1192] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jul 6 13:34:32 2022] PUT /identity/v3/projects/405ed24ac9c84d4da611d2f05b1efe42/users/91d2f93c10dc4160bfae808333c3edf6/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:34:34.458251 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908ef06-f1f3-4924-b088-1cf33b64f5d6 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:34.458831 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908ef06-f1f3-4924-b088-1cf33b64f5d6 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:34.458831 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908ef06-f1f3-4924-b088-1cf33b64f5d6 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:34.460176 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 592/1193] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:34:34.466663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc60e5c-3d3c-427d-8a7a-006ce3cb06a1 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:34.467189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc60e5c-3d3c-427d-8a7a-006ce3cb06a1 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:34.467682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc60e5c-3d3c-427d-8a7a-006ce3cb06a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:34.488754 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-4bc60e5c-3d3c-427d-8a7a-006ce3cb06a1 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:34:34.490775 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-4bc60e5c-3d3c-427d-8a7a-006ce3cb06a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:34:34.571274 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 602/1194] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:34:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:34:34.596499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c908ef06-f1f3-4924-b088-1cf33b64f5d6 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:34.665586 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:34.695699 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c908ef06-f1f3-4924-b088-1cf33b64f5d6 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:34.728747 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d68864-bdfd-4c54-b30b-741111d6be73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:34.729553 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d68864-bdfd-4c54-b30b-741111d6be73 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:34.730543 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d68864-bdfd-4c54-b30b-741111d6be73 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:34.730885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d68864-bdfd-4c54-b30b-741111d6be73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:34.740656 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d68864-bdfd-4c54-b30b-741111d6be73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:34:34.748420 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-27d68864-bdfd-4c54-b30b-741111d6be73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:34:34.748420 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:34:34.748420 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:34:34.748420 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:34:34.748420 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:34:34.748420 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:34:34.748420 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:34:34.748420 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:34:34.748420 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:34:34.748420 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:34:34.748420 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:34:34.748420 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:34:34.748420 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:34:34.748420 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:34:34.748420 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:34:34.748420 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:34:34.749538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:34:34.749538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:34:34.749538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:34:34.749538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:34:34.749538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:34:34.749538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:34:34.749538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:34:34.749538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:34:34.749538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:34:34.749538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:34:34.749538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:34:34.749538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:34:34.749538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:34:34.749538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:34:34.749538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:34:34.749538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:34:34.750397 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:34:34.750397 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:34:34.750397 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:34:34.750397 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:34:34.750397 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:34:34.750397 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:34:34.750397 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:34:34.750397 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:34:34.750397 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:34:34.750397 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:34:34.750397 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:34:34.750397 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:34:34.750397 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 593/1195] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:34:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:34:34.755669 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc60e5c-3d3c-427d-8a7a-006ce3cb06a1 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:34.843671 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:34.865820 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc60e5c-3d3c-427d-8a7a-006ce3cb06a1 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:34.899034 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428fd5f-8d8b-42a8-bcb5-87fd39b4643f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:34.899691 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428fd5f-8d8b-42a8-bcb5-87fd39b4643f None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:34.899971 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428fd5f-8d8b-42a8-bcb5-87fd39b4643f None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:34.900243 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428fd5f-8d8b-42a8-bcb5-87fd39b4643f None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:34.900977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0428fd5f-8d8b-42a8-bcb5-87fd39b4643f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:34:34.901308 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0428fd5f-8d8b-42a8-bcb5-87fd39b4643f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:34.903290 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0428fd5f-8d8b-42a8-bcb5-87fd39b4643f None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:34.910999 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 603/1196] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:34:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:34:34.916839 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d68864-bdfd-4c54-b30b-741111d6be73 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:35.014081 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d68864-bdfd-4c54-b30b-741111d6be73 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:35.041811 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:35.047566 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6992f2f-5535-4586-aa1f-18df5e28f89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:35.048690 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6992f2f-5535-4586-aa1f-18df5e28f89e None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:35.048975 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6992f2f-5535-4586-aa1f-18df5e28f89e None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:35.049287 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6992f2f-5535-4586-aa1f-18df5e28f89e None admin] PATH_INFO: `/v3/users/91d2f93c10dc4160bfae808333c3edf6` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:35.050466 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6992f2f-5535-4586-aa1f-18df5e28f89e None admin] RBAC: Authorizing `identity:get_user(user_id=91d2f93c10dc4160bfae808333c3edf6)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:35.051899 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6992f2f-5535-4586-aa1f-18df5e28f89e None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:35.053569 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 594/1197] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jul 6 13:34:34 2022] GET /identity/v3/users/91d2f93c10dc4160bfae808333c3edf6 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:34:35.059522 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428fd5f-8d8b-42a8-bcb5-87fd39b4643f None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:35.171558 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428fd5f-8d8b-42a8-bcb5-87fd39b4643f None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:35.204774 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d090ad6-12d5-454d-a32a-2ff752346ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:35.205508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d090ad6-12d5-454d-a32a-2ff752346ea8 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:35.205794 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d090ad6-12d5-454d-a32a-2ff752346ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:35.206072 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d090ad6-12d5-454d-a32a-2ff752346ea8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:35.216407 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:35.271451 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d090ad6-12d5-454d-a32a-2ff752346ea8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:35.272876 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d090ad6-12d5-454d-a32a-2ff752346ea8 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:35.368181 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 604/1198] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:34:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 13:34:35.374903 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6992f2f-5535-4586-aa1f-18df5e28f89e None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:35.475726 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6992f2f-5535-4586-aa1f-18df5e28f89e None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:35.508083 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704b8ea-fcf3-4212-9df3-7665a78894b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:35.508932 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704b8ea-fcf3-4212-9df3-7665a78894b0 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:35.509243 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704b8ea-fcf3-4212-9df3-7665a78894b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:35.509517 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704b8ea-fcf3-4212-9df3-7665a78894b0 None admin] PATH_INFO: `/v3/projects/405ed24ac9c84d4da611d2f05b1efe42` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:35.510675 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3704b8ea-fcf3-4212-9df3-7665a78894b0 None admin] RBAC: Authorizing `identity:get_project(project_id=405ed24ac9c84d4da611d2f05b1efe42)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:35.512819 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3704b8ea-fcf3-4212-9df3-7665a78894b0 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:35.514526 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 595/1199] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jul 6 13:34:35 2022] GET /identity/v3/projects/405ed24ac9c84d4da611d2f05b1efe42 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:34:35.521124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d090ad6-12d5-454d-a32a-2ff752346ea8 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:34:35.563566 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:35.619429 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d090ad6-12d5-454d-a32a-2ff752346ea8 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:34:35.653200 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd022cdc-1068-43af-9fd7-10f174f9b9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:34:35.653838 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd022cdc-1068-43af-9fd7-10f174f9b9f5 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:34:35.654121 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd022cdc-1068-43af-9fd7-10f174f9b9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:34:35.654396 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd022cdc-1068-43af-9fd7-10f174f9b9f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:34:35.655219 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd022cdc-1068-43af-9fd7-10f174f9b9f5 None admin] RBAC: Adding query filter params (role.id=795daa2c78354fd48e4a741e1182cc98, scope.project.id=405ed24ac9c84d4da611d2f05b1efe42, user.id=91d2f93c10dc4160bfae808333c3edf6) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:34:35.655528 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd022cdc-1068-43af-9fd7-10f174f9b9f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:34:35.656932 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd022cdc-1068-43af-9fd7-10f174f9b9f5 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:34:35.662527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 605/1200] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:34:35 2022] GET /identity/v3/role_assignments?user.id=91d2f93c10dc4160bfae808333c3edf6&scope.project.id=405ed24ac9c84d4da611d2f05b1efe42&role.id=795daa2c78354fd48e4a741e1182cc98 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:34:45.504077 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=66 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:34:45.660041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=48 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:31.159187 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c61721-801a-4fe5-8d81-a838763ab2f6 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:31.159861 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c61721-801a-4fe5-8d81-a838763ab2f6 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:31.159861 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c61721-801a-4fe5-8d81-a838763ab2f6 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:31.161623 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 596/1201] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:35:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:35:31.168565 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4e202-3879-403c-b471-a9607c16175d None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:31.169091 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4e202-3879-403c-b471-a9607c16175d None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:31.169182 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4e202-3879-403c-b471-a9607c16175d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:31.191428 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-b9b4e202-3879-403c-b471-a9607c16175d None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:35:31.192922 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-b9b4e202-3879-403c-b471-a9607c16175d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:35:31.284462 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 606/1202] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:35:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:35:31.312354 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c61721-801a-4fe5-8d81-a838763ab2f6 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:31.451342 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c61721-801a-4fe5-8d81-a838763ab2f6 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:31.497622 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d3b19-278e-4645-aa60-0e2a2175fe37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:31.498904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d3b19-278e-4645-aa60-0e2a2175fe37 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:31.499358 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d3b19-278e-4645-aa60-0e2a2175fe37 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:31.499835 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d3b19-278e-4645-aa60-0e2a2175fe37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:31.513699 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d3b19-278e-4645-aa60-0e2a2175fe37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:31.515782 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d3b19-278e-4645-aa60-0e2a2175fe37 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:31.541809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-752d3b19-278e-4645-aa60-0e2a2175fe37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:35:31.541809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:35:31.541809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:35:31.541809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:35:31.541809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:35:31.541809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:35:31.541809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:35:31.541809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:35:31.541809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:35:31.541809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:35:31.541809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:35:31.541809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:35:31.541809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:35:31.541809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:35:31.541809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:35:31.543152 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:35:31.543152 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:35:31.543152 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:35:31.543152 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:35:31.543152 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:35:31.543152 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:35:31.543152 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:35:31.543152 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:35:31.543152 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:35:31.543152 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:35:31.543152 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:35:31.543152 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:35:31.543152 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:35:31.543152 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:35:31.543152 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:35:31.543152 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:35:31.543152 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:35:31.544786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:35:31.544786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:35:31.544786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:35:31.544786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:35:31.544786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 597/1203] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:35:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:35:31.550648 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4e202-3879-403c-b471-a9607c16175d None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:31.704754 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4e202-3879-403c-b471-a9607c16175d None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:31.740689 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:31.749150 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b466ad-f085-4028-babe-a11f8b4d6870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:31.749830 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b466ad-f085-4028-babe-a11f8b4d6870 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:31.750111 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b466ad-f085-4028-babe-a11f8b4d6870 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:31.750386 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b466ad-f085-4028-babe-a11f8b4d6870 None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:31.751120 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b466ad-f085-4028-babe-a11f8b4d6870 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:31.751424 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b466ad-f085-4028-babe-a11f8b4d6870 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:31.753077 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b466ad-f085-4028-babe-a11f8b4d6870 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:31.777079 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 607/1204] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:35:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:35:31.779912 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d3b19-278e-4645-aa60-0e2a2175fe37 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:31.860288 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:31.921138 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d3b19-278e-4645-aa60-0e2a2175fe37 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:31.963550 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595af881-0573-4fae-8caa-7f369a937de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:31.963949 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595af881-0573-4fae-8caa-7f369a937de0 None admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:31.964334 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595af881-0573-4fae-8caa-7f369a937de0 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:31.965431 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595af881-0573-4fae-8caa-7f369a937de0 None admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:31.965431 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595af881-0573-4fae-8caa-7f369a937de0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:31.967077 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595af881-0573-4fae-8caa-7f369a937de0 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:31.995165 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 598/1205] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jul 6 13:35:31 2022] POST /identity/v3/users => generated 251 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:35:34.178880 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caef3c5-f3c5-4065-83ed-c2f402822629 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:34.179459 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caef3c5-f3c5-4065-83ed-c2f402822629 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:34.179459 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caef3c5-f3c5-4065-83ed-c2f402822629 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:34.181493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 608/1206] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:35:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:35:34.191712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85c188-5059-40ae-b158-19ba2a83a281 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:34.192281 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85c188-5059-40ae-b158-19ba2a83a281 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:34.192281 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85c188-5059-40ae-b158-19ba2a83a281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:34.212623 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-fb85c188-5059-40ae-b158-19ba2a83a281 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:35:34.214076 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-fb85c188-5059-40ae-b158-19ba2a83a281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:35:34.293358 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 599/1207] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:35:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:35:34.320272 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caef3c5-f3c5-4065-83ed-c2f402822629 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:34.449107 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caef3c5-f3c5-4065-83ed-c2f402822629 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:34.484748 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776cc620-a4fc-4d36-9970-4453f5dd9296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:34.485614 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776cc620-a4fc-4d36-9970-4453f5dd9296 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:34.485901 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776cc620-a4fc-4d36-9970-4453f5dd9296 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:34.486230 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776cc620-a4fc-4d36-9970-4453f5dd9296 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:34.500155 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776cc620-a4fc-4d36-9970-4453f5dd9296 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:35:34.507815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-776cc620-a4fc-4d36-9970-4453f5dd9296 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:35:34.507815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:35:34.507815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:35:34.507815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:35:34.507815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:35:34.507815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:35:34.507815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:35:34.507815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:35:34.507815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:35:34.507815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:35:34.507815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:35:34.507815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:35:34.507815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:35:34.507815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:35:34.507815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:35:34.507815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:35:34.509491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:35:34.509491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:35:34.509491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:35:34.509491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:35:34.509491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:35:34.509491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:35:34.509491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:35:34.509491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:35:34.509491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:35:34.509491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:35:34.509491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:35:34.509491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:35:34.509491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:35:34.509491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:35:34.509491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:35:34.509491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:35:34.510357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:35:34.510357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:35:34.510357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:35:34.510357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:35:34.510357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:35:34.510357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:35:34.510357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:35:34.510357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:35:34.510357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:35:34.510357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:35:34.510357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:35:34.510357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:35:34.510357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 609/1208] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jul 6 13:35:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:35:34.515447 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85c188-5059-40ae-b158-19ba2a83a281 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:34.562199 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:34.628426 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85c188-5059-40ae-b158-19ba2a83a281 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:34.671523 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecf8bf-d1ea-43e1-b721-c99d8aa46fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:34.672265 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecf8bf-d1ea-43e1-b721-c99d8aa46fb9 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:34.672806 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecf8bf-d1ea-43e1-b721-c99d8aa46fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:34.673126 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecf8bf-d1ea-43e1-b721-c99d8aa46fb9 None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:34.673884 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ecf8bf-d1ea-43e1-b721-c99d8aa46fb9 None admin] RBAC: Adding query filter params (name=service) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:34.674191 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ecf8bf-d1ea-43e1-b721-c99d8aa46fb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:34.675560 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ecf8bf-d1ea-43e1-b721-c99d8aa46fb9 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:34.690901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 600/1209] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jul 6 13:35:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:35:34.692197 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776cc620-a4fc-4d36-9970-4453f5dd9296 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:34.755118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:34.824839 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776cc620-a4fc-4d36-9970-4453f5dd9296 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:34.876976 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533014a-b5b6-41ad-8528-9ba232ce98ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:34.877919 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533014a-b5b6-41ad-8528-9ba232ce98ad None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:34.878271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533014a-b5b6-41ad-8528-9ba232ce98ad None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:34.878577 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533014a-b5b6-41ad-8528-9ba232ce98ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:34.893145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1533014a-b5b6-41ad-8528-9ba232ce98ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:34.895483 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1533014a-b5b6-41ad-8528-9ba232ce98ad None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:34.919257 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-1533014a-b5b6-41ad-8528-9ba232ce98ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:35:34.919257 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:35:34.919257 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:35:34.919257 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:35:34.919257 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:35:34.919257 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:35:34.919257 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:35:34.919257 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:35:34.919257 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:35:34.919257 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:35:34.919257 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:35:34.919257 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:35:34.919257 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:35:34.919257 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:35:34.919257 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:35:34.920271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:35:34.920271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:35:34.920271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:35:34.920271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:35:34.920271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:35:34.920271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:35:34.920271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:35:34.920271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:35:34.920271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:35:34.920271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:35:34.920271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:35:34.920271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:35:34.920271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:35:34.920271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:35:34.920271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:35:34.920271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:35:34.920271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:35:34.921257 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:35:34.921257 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:35:34.921257 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:35:34.921257 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:35:34.921257 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 610/1210] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:35:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:35:34.926696 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecf8bf-d1ea-43e1-b721-c99d8aa46fb9 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:35.055341 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecf8bf-d1ea-43e1-b721-c99d8aa46fb9 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:35.056940 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:35.096427 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c588f98a-16af-412b-abee-c9c35ee0bcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:35.097118 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c588f98a-16af-412b-abee-c9c35ee0bcb9 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:35.097401 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c588f98a-16af-412b-abee-c9c35ee0bcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:35.097670 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c588f98a-16af-412b-abee-c9c35ee0bcb9 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:35.098394 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c588f98a-16af-412b-abee-c9c35ee0bcb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:35.098704 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c588f98a-16af-412b-abee-c9c35ee0bcb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:35.100076 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c588f98a-16af-412b-abee-c9c35ee0bcb9 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:35.112883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 601/1211] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:35:35.119230 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533014a-b5b6-41ad-8528-9ba232ce98ad None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:35.234803 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533014a-b5b6-41ad-8528-9ba232ce98ad None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:35.237179 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:35.274600 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a313faf-d533-4b06-bfb0-8e9c3b4aa4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:35.275407 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a313faf-d533-4b06-bfb0-8e9c3b4aa4b0 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:35.275695 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a313faf-d533-4b06-bfb0-8e9c3b4aa4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:35.275976 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a313faf-d533-4b06-bfb0-8e9c3b4aa4b0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:35.284555 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a313faf-d533-4b06-bfb0-8e9c3b4aa4b0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:35.288307 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a313faf-d533-4b06-bfb0-8e9c3b4aa4b0 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:35.305851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-3a313faf-d533-4b06-bfb0-8e9c3b4aa4b0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 06 13:35:35.305851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:35:35.305851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:35:35.305851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:35:35.305851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:35:35.305851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:35:35.305851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:35:35.305851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:35:35.305851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:35:35.305851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:35:35.305851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:35:35.305851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:35:35.305851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:35:35.305851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:35:35.305851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:35:35.305851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:35:35.307255 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:35:35.307255 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:35:35.307255 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:35:35.307255 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:35:35.307255 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:35:35.307255 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:35:35.307255 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:35:35.307255 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:35:35.307255 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:35:35.307255 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:35:35.307255 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:35:35.307255 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:35:35.307255 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:35:35.307255 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:35:35.307255 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:35:35.307255 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:35:35.307255 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:35:35.307255 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:35:35.308221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:35:35.308221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:35:35.308221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:35:35.308221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:35:35.308221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:35:35.308221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:35:35.308221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:35:35.308221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:35:35.308221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 06 13:35:35.308221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:35:35.308221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 611/1212] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:35:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:35:35.312982 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c588f98a-16af-412b-abee-c9c35ee0bcb9 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:35.429218 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c588f98a-16af-412b-abee-c9c35ee0bcb9 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:35.468895 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c11b07-b3fd-4592-9183-068b2da30306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:35.469627 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c11b07-b3fd-4592-9183-068b2da30306 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:35.470002 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c11b07-b3fd-4592-9183-068b2da30306 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:35.470254 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c11b07-b3fd-4592-9183-068b2da30306 None admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:35.471063 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c11b07-b3fd-4592-9183-068b2da30306 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:35.471374 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c11b07-b3fd-4592-9183-068b2da30306 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:35.472819 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c11b07-b3fd-4592-9183-068b2da30306 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:35.491733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 602/1213] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Jul 6 13:35:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:35:35.498154 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a313faf-d533-4b06-bfb0-8e9c3b4aa4b0 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:35.630582 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a313faf-d533-4b06-bfb0-8e9c3b4aa4b0 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:35.672051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f378220e-dd1e-46f9-bc7b-5f0b4fe1d9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:35.672945 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378220e-dd1e-46f9-bc7b-5f0b4fe1d9ce None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:35.673310 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378220e-dd1e-46f9-bc7b-5f0b4fe1d9ce None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:35.673593 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378220e-dd1e-46f9-bc7b-5f0b4fe1d9ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:35.713848 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:35.769206 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f378220e-dd1e-46f9-bc7b-5f0b4fe1d9ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:35.771278 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f378220e-dd1e-46f9-bc7b-5f0b4fe1d9ce None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:35.898425 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 612/1214] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:35:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 13:35:35.900401 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c11b07-b3fd-4592-9183-068b2da30306 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:35.928707 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:36.042785 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c11b07-b3fd-4592-9183-068b2da30306 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:36.091994 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e802d76-c330-46ad-9673-857b0336e934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:36.093091 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e802d76-c330-46ad-9673-857b0336e934 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:36.093616 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e802d76-c330-46ad-9673-857b0336e934 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:36.095023 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e802d76-c330-46ad-9673-857b0336e934 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:36.109177 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e802d76-c330-46ad-9673-857b0336e934 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:36.110987 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e802d76-c330-46ad-9673-857b0336e934 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:36.132504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-7e802d76-c330-46ad-9673-857b0336e934 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:35:36.132504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:35:36.132504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:35:36.132504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:35:36.132504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:35:36.132504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:35:36.132504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:35:36.132504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:35:36.132504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:35:36.132504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:35:36.132504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:35:36.132504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:35:36.132504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:35:36.132504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:35:36.132504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:35:36.134673 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:35:36.134673 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:35:36.134673 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:35:36.134673 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:35:36.134673 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:35:36.134673 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:35:36.134673 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:35:36.134673 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:35:36.134673 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:35:36.134673 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:35:36.134673 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:35:36.134673 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:35:36.134673 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:35:36.134673 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:35:36.134673 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:35:36.134673 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:35:36.134673 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:35:36.135647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:35:36.135647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:35:36.135647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:35:36.135647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:35:36.135647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 603/1215] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:35:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:35:36.141814 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f378220e-dd1e-46f9-bc7b-5f0b4fe1d9ce None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:36.224443 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:36.285709 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f378220e-dd1e-46f9-bc7b-5f0b4fe1d9ce None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:36.337243 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e3c4d-44ee-4a81-9517-f16b12054eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:36.337954 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e3c4d-44ee-4a81-9517-f16b12054eac None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:36.338236 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e3c4d-44ee-4a81-9517-f16b12054eac None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:36.338510 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e3c4d-44ee-4a81-9517-f16b12054eac None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:36.341212 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32e3c4d-44ee-4a81-9517-f16b12054eac None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:36.341212 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32e3c4d-44ee-4a81-9517-f16b12054eac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:36.341388 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32e3c4d-44ee-4a81-9517-f16b12054eac None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:36.357264 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 613/1216] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:35:36.363278 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e802d76-c330-46ad-9673-857b0336e934 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:36.467505 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:36.505440 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e802d76-c330-46ad-9673-857b0336e934 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:36.549145 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dccb082-4e6e-4be3-9d66-9e6dbfd6887e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:36.549937 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dccb082-4e6e-4be3-9d66-9e6dbfd6887e None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:36.550228 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dccb082-4e6e-4be3-9d66-9e6dbfd6887e None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:36.550547 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dccb082-4e6e-4be3-9d66-9e6dbfd6887e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:36.560429 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dccb082-4e6e-4be3-9d66-9e6dbfd6887e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:36.562509 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dccb082-4e6e-4be3-9d66-9e6dbfd6887e None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:36.580130 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-3dccb082-4e6e-4be3-9d66-9e6dbfd6887e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:35:36.580130 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:35:36.580130 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:35:36.580130 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:35:36.580130 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:35:36.580130 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:35:36.580130 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:35:36.580130 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:35:36.580130 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:35:36.580130 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:35:36.580130 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:35:36.580130 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:35:36.580130 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:35:36.580130 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:35:36.580130 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:35:36.587373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:35:36.587373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:35:36.587373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:35:36.587373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:35:36.587373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:35:36.587373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:35:36.587373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:35:36.587373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:35:36.587373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:35:36.587373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:35:36.587373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:35:36.587373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:35:36.587373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:35:36.587373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:35:36.587373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:35:36.587373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:35:36.587373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:35:36.588391 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:35:36.588391 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:35:36.588391 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:35:36.588391 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:35:36.588391 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:35:36.588391 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:35:36.588391 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:35:36.588391 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:35:36.588391 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 604/1217] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:35:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:35:36.588990 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e3c4d-44ee-4a81-9517-f16b12054eac None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:36.738874 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e3c4d-44ee-4a81-9517-f16b12054eac None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:36.780299 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:36.794248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46727229-aba2-4ca1-9f4e-ff1ad2c14521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:36.794920 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46727229-aba2-4ca1-9f4e-ff1ad2c14521 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:36.795203 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46727229-aba2-4ca1-9f4e-ff1ad2c14521 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:36.795481 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46727229-aba2-4ca1-9f4e-ff1ad2c14521 None admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:36.796285 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46727229-aba2-4ca1-9f4e-ff1ad2c14521 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:36.796600 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46727229-aba2-4ca1-9f4e-ff1ad2c14521 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:36.798327 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46727229-aba2-4ca1-9f4e-ff1ad2c14521 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:36.812451 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 614/1218] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:35:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:35:36.818916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dccb082-4e6e-4be3-9d66-9e6dbfd6887e None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:36.947991 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dccb082-4e6e-4be3-9d66-9e6dbfd6887e None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:36.990842 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:36.997692 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64de4b0-545a-4fda-b72f-a8c1586b4981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:36.998575 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64de4b0-545a-4fda-b72f-a8c1586b4981 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:36.998875 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64de4b0-545a-4fda-b72f-a8c1586b4981 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:36.999137 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64de4b0-545a-4fda-b72f-a8c1586b4981 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:37.000565 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64de4b0-545a-4fda-b72f-a8c1586b4981 None admin] RBAC: Adding query filter params (role.id=a6853d485f7942259b3646de3f386852, scope.project.id=bfe6c3d8da2743be9609184db2506303, user.id=890d2ae150d647a18dae25159cf97484) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:37.001392 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64de4b0-545a-4fda-b72f-a8c1586b4981 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:37.004752 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64de4b0-545a-4fda-b72f-a8c1586b4981 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:37.025033 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 605/1219] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:35:36 2022] GET /identity/v3/role_assignments?user.id=890d2ae150d647a18dae25159cf97484&scope.project.id=bfe6c3d8da2743be9609184db2506303&role.id=a6853d485f7942259b3646de3f386852 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:35:39.161353 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017c4be-d9a8-427f-96f9-a983a352c98a None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:39.161979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017c4be-d9a8-427f-96f9-a983a352c98a None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:39.161979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017c4be-d9a8-427f-96f9-a983a352c98a None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:39.163362 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 615/1220] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:35:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:35:39.169183 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f21110-e226-4b74-bf9b-3743e160678d None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:39.169609 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f21110-e226-4b74-bf9b-3743e160678d None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:39.169750 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f21110-e226-4b74-bf9b-3743e160678d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:39.187493 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-b9f21110-e226-4b74-bf9b-3743e160678d None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:35:39.188755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-b9f21110-e226-4b74-bf9b-3743e160678d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:35:39.264048 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 606/1221] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:35:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:35:39.287443 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017c4be-d9a8-427f-96f9-a983a352c98a None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:39.407327 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017c4be-d9a8-427f-96f9-a983a352c98a None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:39.446937 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086daac2-4d74-49f7-9bd3-c3b1c56cd5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:39.447810 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086daac2-4d74-49f7-9bd3-c3b1c56cd5f2 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:39.448125 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086daac2-4d74-49f7-9bd3-c3b1c56cd5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:39.448406 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086daac2-4d74-49f7-9bd3-c3b1c56cd5f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:39.459272 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086daac2-4d74-49f7-9bd3-c3b1c56cd5f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:35:39.467251 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-086daac2-4d74-49f7-9bd3-c3b1c56cd5f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:35:39.467251 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:35:39.467251 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:35:39.467251 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:35:39.467251 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:35:39.467251 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:35:39.467251 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:35:39.467251 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:35:39.467251 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:35:39.467251 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:35:39.467251 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:35:39.467251 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:35:39.467251 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:35:39.467251 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:35:39.467251 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:35:39.467251 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:35:39.471704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:35:39.471704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:35:39.471704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:35:39.471704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:35:39.471704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:35:39.471704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:35:39.471704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:35:39.471704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:35:39.471704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:35:39.471704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:35:39.471704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:35:39.471704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:35:39.471704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:35:39.471704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:35:39.471704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:35:39.471704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:35:39.472540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:35:39.472540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:35:39.472540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:35:39.472540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:35:39.472540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:35:39.472540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:35:39.472540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:35:39.472540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:35:39.472540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:35:39.472540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:35:39.472540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:35:39.472540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:35:39.472540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 616/1222] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jul 6 13:35:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:35:39.474745 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f21110-e226-4b74-bf9b-3743e160678d None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:39.579471 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f21110-e226-4b74-bf9b-3743e160678d None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:39.604589 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:39.617024 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce36097-82b3-46b1-ba5f-f47523b51c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:39.617726 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce36097-82b3-46b1-ba5f-f47523b51c33 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:39.618013 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce36097-82b3-46b1-ba5f-f47523b51c33 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:39.618295 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce36097-82b3-46b1-ba5f-f47523b51c33 None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:39.619108 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce36097-82b3-46b1-ba5f-f47523b51c33 None admin] RBAC: Adding query filter params (name=service) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:39.619426 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce36097-82b3-46b1-ba5f-f47523b51c33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:39.620801 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce36097-82b3-46b1-ba5f-f47523b51c33 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:39.634482 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 607/1223] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jul 6 13:35:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:35:39.636749 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086daac2-4d74-49f7-9bd3-c3b1c56cd5f2 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:39.684216 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:39.742709 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086daac2-4d74-49f7-9bd3-c3b1c56cd5f2 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:39.778447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f221cdb-4241-4bb7-88fb-7be5f4afe925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:39.779229 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f221cdb-4241-4bb7-88fb-7be5f4afe925 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:39.779550 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f221cdb-4241-4bb7-88fb-7be5f4afe925 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:39.779832 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f221cdb-4241-4bb7-88fb-7be5f4afe925 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:39.790630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f221cdb-4241-4bb7-88fb-7be5f4afe925 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:39.792066 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f221cdb-4241-4bb7-88fb-7be5f4afe925 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:39.810194 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-9f221cdb-4241-4bb7-88fb-7be5f4afe925 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:35:39.810194 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:35:39.810194 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:35:39.810194 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:35:39.810194 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:35:39.810194 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:35:39.810194 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:35:39.810194 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:35:39.810194 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:35:39.810194 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:35:39.810194 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:35:39.810194 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:35:39.810194 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:35:39.810194 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:35:39.810194 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:35:39.811309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:35:39.811309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:35:39.811309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:35:39.811309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:35:39.811309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:35:39.811309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:35:39.811309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:35:39.811309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:35:39.811309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:35:39.811309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:35:39.811309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:35:39.811309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:35:39.811309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:35:39.811309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:35:39.811309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:35:39.811309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:35:39.811309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:35:39.812239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:35:39.812239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:35:39.812239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:35:39.812239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:35:39.812239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 617/1224] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:35:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:35:39.817320 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce36097-82b3-46b1-ba5f-f47523b51c33 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:39.921891 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce36097-82b3-46b1-ba5f-f47523b51c33 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:39.956741 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c1963-fe66-4797-ab9e-7af0d1c55d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:39.957860 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c1963-fe66-4797-ab9e-7af0d1c55d07 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:39.958184 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c1963-fe66-4797-ab9e-7af0d1c55d07 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:39.958487 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c1963-fe66-4797-ab9e-7af0d1c55d07 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:39.959220 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c1963-fe66-4797-ab9e-7af0d1c55d07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:39.959525 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c1963-fe66-4797-ab9e-7af0d1c55d07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:39.961392 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c1963-fe66-4797-ab9e-7af0d1c55d07 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:39.975034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 608/1225] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:35:39.981342 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f221cdb-4241-4bb7-88fb-7be5f4afe925 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:40.086846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:40.097089 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f221cdb-4241-4bb7-88fb-7be5f4afe925 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:40.133466 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0a125-2ca5-4ff6-8fd8-7c9b1af60e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:40.134382 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0a125-2ca5-4ff6-8fd8-7c9b1af60e04 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:40.134664 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0a125-2ca5-4ff6-8fd8-7c9b1af60e04 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:40.134936 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0a125-2ca5-4ff6-8fd8-7c9b1af60e04 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:40.144002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a0a125-2ca5-4ff6-8fd8-7c9b1af60e04 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:40.145666 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a0a125-2ca5-4ff6-8fd8-7c9b1af60e04 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:40.160814 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-61a0a125-2ca5-4ff6-8fd8-7c9b1af60e04 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 06 13:35:40.160814 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:35:40.160814 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:35:40.160814 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:35:40.160814 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:35:40.160814 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:35:40.160814 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:35:40.160814 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:35:40.160814 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:35:40.160814 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:35:40.160814 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:35:40.160814 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:35:40.160814 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:35:40.160814 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:35:40.160814 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:35:40.160814 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:35:40.162327 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:35:40.162327 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:35:40.162327 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:35:40.162327 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:35:40.162327 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:35:40.162327 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:35:40.162327 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:35:40.162327 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:35:40.162327 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:35:40.162327 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:35:40.162327 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:35:40.162327 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:35:40.162327 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:35:40.162327 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:35:40.162327 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:35:40.162327 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:35:40.162327 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:35:40.162327 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:35:40.163286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:35:40.163286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:35:40.163286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:35:40.163286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:35:40.163286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:35:40.163286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:35:40.163286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:35:40.163286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:35:40.163286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 06 13:35:40.163286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:35:40.163286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 618/1226] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:35:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:35:40.168975 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c1963-fe66-4797-ab9e-7af0d1c55d07 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:40.216924 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:40.274429 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c1963-fe66-4797-ab9e-7af0d1c55d07 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:40.314478 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e585431-1393-4fc5-a940-5c4373961ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:40.315304 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e585431-1393-4fc5-a940-5c4373961ac1 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:40.315664 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e585431-1393-4fc5-a940-5c4373961ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:40.315990 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e585431-1393-4fc5-a940-5c4373961ac1 None admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:40.316990 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e585431-1393-4fc5-a940-5c4373961ac1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:40.317327 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e585431-1393-4fc5-a940-5c4373961ac1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:40.318961 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e585431-1393-4fc5-a940-5c4373961ac1 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:40.337303 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 609/1227] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Jul 6 13:35:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:35:40.343055 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0a125-2ca5-4ff6-8fd8-7c9b1af60e04 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:40.459786 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0a125-2ca5-4ff6-8fd8-7c9b1af60e04 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:40.499733 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b428a-49ae-4d80-9a42-8f113e77e533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:40.500664 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b428a-49ae-4d80-9a42-8f113e77e533 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:40.501067 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b428a-49ae-4d80-9a42-8f113e77e533 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:40.501459 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b428a-49ae-4d80-9a42-8f113e77e533 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:40.514536 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b428a-49ae-4d80-9a42-8f113e77e533 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:40.516345 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b428a-49ae-4d80-9a42-8f113e77e533 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:40.538175 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-ec3b428a-49ae-4d80-9a42-8f113e77e533 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:35:40.538175 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:35:40.538175 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:35:40.538175 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:35:40.538175 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:35:40.538175 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:35:40.538175 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:35:40.538175 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:35:40.538175 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:35:40.538175 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:35:40.538175 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:35:40.538175 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:35:40.538175 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:35:40.538175 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:35:40.538175 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:35:40.540162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:35:40.540162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:35:40.540162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:35:40.540162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:35:40.540162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:35:40.540162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:35:40.540162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:35:40.540162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:35:40.540162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:35:40.540162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:35:40.540162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:35:40.540162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:35:40.540162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:35:40.540162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:35:40.540162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:35:40.540162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:35:40.540162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:35:40.541456 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:35:40.541456 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:35:40.541456 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:35:40.541456 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:35:40.541456 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 619/1228] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:35:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:35:40.546367 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e585431-1393-4fc5-a940-5c4373961ac1 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:40.659982 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e585431-1393-4fc5-a940-5c4373961ac1 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:40.662363 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:40.699835 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f43aa-8c9e-44e9-ad43-7d6fdd9a51d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:40.700582 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f43aa-8c9e-44e9-ad43-7d6fdd9a51d9 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:40.700865 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f43aa-8c9e-44e9-ad43-7d6fdd9a51d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:40.701166 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f43aa-8c9e-44e9-ad43-7d6fdd9a51d9 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:40.701887 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f43aa-8c9e-44e9-ad43-7d6fdd9a51d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:40.702193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f43aa-8c9e-44e9-ad43-7d6fdd9a51d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:40.703721 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f43aa-8c9e-44e9-ad43-7d6fdd9a51d9 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:40.716531 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 610/1229] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:35:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:35:40.722611 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b428a-49ae-4d80-9a42-8f113e77e533 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:40.735143 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:40.839877 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b428a-49ae-4d80-9a42-8f113e77e533 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:40.875941 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3721a6-90f3-4def-b62f-650d1a32834b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:40.876813 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3721a6-90f3-4def-b62f-650d1a32834b None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:40.877195 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3721a6-90f3-4def-b62f-650d1a32834b None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:40.877537 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3721a6-90f3-4def-b62f-650d1a32834b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:40.884962 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3721a6-90f3-4def-b62f-650d1a32834b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:40.886425 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3721a6-90f3-4def-b62f-650d1a32834b None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:40.902846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-2b3721a6-90f3-4def-b62f-650d1a32834b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:35:40.902846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:35:40.902846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:35:40.902846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:35:40.902846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:35:40.902846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:35:40.902846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:35:40.902846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:35:40.902846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:35:40.902846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:35:40.902846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:35:40.902846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:35:40.902846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:35:40.902846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:35:40.902846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:35:40.903977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:35:40.903977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:35:40.903977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:35:40.903977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:35:40.903977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:35:40.903977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:35:40.903977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:35:40.903977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:35:40.903977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:35:40.903977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:35:40.903977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:35:40.903977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:35:40.903977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:35:40.903977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:35:40.903977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:35:40.903977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:35:40.903977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:35:40.905261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:35:40.905261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:35:40.905261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:35:40.905261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:35:40.905261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:35:40.905261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:35:40.905261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:35:40.905261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:35:40.905261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 620/1230] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:35:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:35:40.910085 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f43aa-8c9e-44e9-ad43-7d6fdd9a51d9 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:41.035500 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f43aa-8c9e-44e9-ad43-7d6fdd9a51d9 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:41.078268 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd33ce0-f6b4-40f9-bc0c-56ca53792b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:41.079014 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd33ce0-f6b4-40f9-bc0c-56ca53792b14 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:41.079305 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd33ce0-f6b4-40f9-bc0c-56ca53792b14 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:41.079601 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd33ce0-f6b4-40f9-bc0c-56ca53792b14 None admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:41.080410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd33ce0-f6b4-40f9-bc0c-56ca53792b14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:41.080694 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd33ce0-f6b4-40f9-bc0c-56ca53792b14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:41.084842 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd33ce0-f6b4-40f9-bc0c-56ca53792b14 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:41.101988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 611/1231] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:35:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:35:41.103564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3721a6-90f3-4def-b62f-650d1a32834b None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:41.175417 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:41.211025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3721a6-90f3-4def-b62f-650d1a32834b None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:41.248011 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66223112-0778-4504-8d9a-70622aa52a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:41.248917 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66223112-0778-4504-8d9a-70622aa52a68 None admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:41.249236 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66223112-0778-4504-8d9a-70622aa52a68 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:41.249515 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66223112-0778-4504-8d9a-70622aa52a68 None admin] PATH_INFO: `/v3/projects/bfe6c3d8da2743be9609184db2506303/users/890d2ae150d647a18dae25159cf97484/roles/a6853d485f7942259b3646de3f386852` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:41.265369 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66223112-0778-4504-8d9a-70622aa52a68 None admin] RBAC: Authorizing `identity:create_grant(project_id=bfe6c3d8da2743be9609184db2506303, user_id=890d2ae150d647a18dae25159cf97484, role_id=a6853d485f7942259b3646de3f386852)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:41.266888 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66223112-0778-4504-8d9a-70622aa52a68 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:41.279614 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 621/1232] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jul 6 13:35:41 2022] PUT /identity/v3/projects/bfe6c3d8da2743be9609184db2506303/users/890d2ae150d647a18dae25159cf97484/roles/a6853d485f7942259b3646de3f386852 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:35:43.385793 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a00a9-f5c1-4ee8-ac32-1ed41b1fba18 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:43.386357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a00a9-f5c1-4ee8-ac32-1ed41b1fba18 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:43.386357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a00a9-f5c1-4ee8-ac32-1ed41b1fba18 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:43.387699 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 612/1233] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:35:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:35:43.392982 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc553ae-ea54-447d-8797-7a19b1443689 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:43.393428 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc553ae-ea54-447d-8797-7a19b1443689 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:43.393567 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc553ae-ea54-447d-8797-7a19b1443689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:43.410060 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-4cc553ae-ea54-447d-8797-7a19b1443689 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:35:43.411241 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-4cc553ae-ea54-447d-8797-7a19b1443689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:35:43.481648 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 622/1234] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:35:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:35:43.505285 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a00a9-f5c1-4ee8-ac32-1ed41b1fba18 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:43.531468 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:43.615256 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a00a9-f5c1-4ee8-ac32-1ed41b1fba18 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:43.651974 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c440f-8751-4b29-8e67-127daec28702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:43.652767 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c440f-8751-4b29-8e67-127daec28702 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:43.653074 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c440f-8751-4b29-8e67-127daec28702 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:43.653346 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c440f-8751-4b29-8e67-127daec28702 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:43.663217 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c440f-8751-4b29-8e67-127daec28702 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:35:43.671581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-336c440f-8751-4b29-8e67-127daec28702 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:35:43.671581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:35:43.671581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:35:43.671581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:35:43.671581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:35:43.671581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:35:43.671581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:35:43.671581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:35:43.671581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:35:43.671581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:35:43.671581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:35:43.671581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:35:43.671581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:35:43.671581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:35:43.671581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:35:43.671581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:35:43.672981 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:35:43.672981 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:35:43.672981 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:35:43.672981 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:35:43.672981 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:35:43.672981 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:35:43.672981 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:35:43.672981 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:35:43.672981 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:35:43.672981 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:35:43.672981 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:35:43.672981 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:35:43.672981 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:35:43.672981 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:35:43.672981 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:35:43.672981 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:35:43.674129 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:35:43.674129 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:35:43.674129 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:35:43.674129 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:35:43.674129 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:35:43.674129 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:35:43.674129 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:35:43.674129 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:35:43.674129 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:35:43.674129 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:35:43.674129 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 13:35:43.674129 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:35:43.678157 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 613/1235] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jul 6 13:35:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:35:43.679180 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc553ae-ea54-447d-8797-7a19b1443689 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:43.725016 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:43.780632 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc553ae-ea54-447d-8797-7a19b1443689 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:43.816485 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd93ad2-d936-4255-a9f1-90281d46d199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:43.817204 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd93ad2-d936-4255-a9f1-90281d46d199 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:43.817491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd93ad2-d936-4255-a9f1-90281d46d199 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:43.817770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd93ad2-d936-4255-a9f1-90281d46d199 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:43.818513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd93ad2-d936-4255-a9f1-90281d46d199 None admin] RBAC: Adding query filter params (name=service) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:43.818823 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd93ad2-d936-4255-a9f1-90281d46d199 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:43.820682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd93ad2-d936-4255-a9f1-90281d46d199 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:43.830972 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 623/1236] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jul 6 13:35:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:35:43.837425 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c440f-8751-4b29-8e67-127daec28702 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:43.943455 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:43.967185 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c440f-8751-4b29-8e67-127daec28702 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:44.015409 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80f357-682d-42b6-a7b1-c93cd7759085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:44.016394 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80f357-682d-42b6-a7b1-c93cd7759085 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:44.016678 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80f357-682d-42b6-a7b1-c93cd7759085 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:44.017058 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80f357-682d-42b6-a7b1-c93cd7759085 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:44.029905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de80f357-682d-42b6-a7b1-c93cd7759085 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:44.031519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de80f357-682d-42b6-a7b1-c93cd7759085 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:44.053032 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-de80f357-682d-42b6-a7b1-c93cd7759085 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:35:44.053032 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:35:44.053032 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:35:44.053032 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:35:44.053032 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:35:44.053032 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:35:44.053032 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:35:44.053032 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:35:44.053032 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:35:44.053032 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:35:44.053032 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:35:44.053032 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:35:44.053032 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:35:44.053032 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:35:44.053032 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:35:44.054078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:35:44.054078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:35:44.054078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:35:44.054078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:35:44.054078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:35:44.054078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:35:44.054078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:35:44.054078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:35:44.054078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:35:44.054078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:35:44.054078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:35:44.054078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:35:44.054078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:35:44.054078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:35:44.054078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:35:44.054078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:35:44.054078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:35:44.055191 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:35:44.055191 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:35:44.055191 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:35:44.055191 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:35:44.061037 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 614/1237] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:35:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 13:35:44.061179 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd93ad2-d936-4255-a9f1-90281d46d199 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:44.181478 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd93ad2-d936-4255-a9f1-90281d46d199 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:44.192237 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:44.221283 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e04e2-ab37-4626-8c01-f51ced150f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:44.221979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e04e2-ab37-4626-8c01-f51ced150f9d None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:44.222338 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e04e2-ab37-4626-8c01-f51ced150f9d None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:44.222757 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e04e2-ab37-4626-8c01-f51ced150f9d None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:44.223676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e04e2-ab37-4626-8c01-f51ced150f9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:44.224148 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e04e2-ab37-4626-8c01-f51ced150f9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:44.225818 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e04e2-ab37-4626-8c01-f51ced150f9d None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:44.238124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 624/1238] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:35:44.243873 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80f357-682d-42b6-a7b1-c93cd7759085 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:44.365077 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80f357-682d-42b6-a7b1-c93cd7759085 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:44.404275 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037102da-b2c3-4621-92d1-60a934596e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:44.405167 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037102da-b2c3-4621-92d1-60a934596e45 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:44.405509 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037102da-b2c3-4621-92d1-60a934596e45 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:44.406002 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037102da-b2c3-4621-92d1-60a934596e45 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:44.409172 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:44.416367 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037102da-b2c3-4621-92d1-60a934596e45 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:44.418403 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037102da-b2c3-4621-92d1-60a934596e45 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:44.437538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-037102da-b2c3-4621-92d1-60a934596e45 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 06 13:35:44.437538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:35:44.437538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:35:44.437538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:35:44.437538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:35:44.437538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:35:44.437538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:35:44.437538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:35:44.437538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:35:44.437538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:35:44.437538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:35:44.437538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:35:44.437538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:35:44.437538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:35:44.437538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:35:44.437538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:35:44.438930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:35:44.438930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:35:44.438930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:35:44.438930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:35:44.438930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:35:44.438930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:35:44.438930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:35:44.438930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:35:44.438930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:35:44.438930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:35:44.438930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:35:44.438930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:35:44.438930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:35:44.438930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:35:44.438930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:35:44.438930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:35:44.438930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:35:44.438930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:35:44.440149 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:35:44.440149 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:35:44.440149 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:35:44.440149 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:35:44.440149 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:35:44.440149 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:35:44.440149 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:35:44.440149 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:35:44.440149 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 06 13:35:44.440149 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:35:44.440149 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 615/1239] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:35:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:35:44.444981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e04e2-ab37-4626-8c01-f51ced150f9d None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:44.555666 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e04e2-ab37-4626-8c01-f51ced150f9d None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:44.594107 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f889e1cf-d55d-4773-b00f-257c82edb55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:44.594845 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889e1cf-d55d-4773-b00f-257c82edb55b None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:44.595139 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889e1cf-d55d-4773-b00f-257c82edb55b None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:44.595456 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889e1cf-d55d-4773-b00f-257c82edb55b None admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:44.596374 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f889e1cf-d55d-4773-b00f-257c82edb55b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:44.596751 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f889e1cf-d55d-4773-b00f-257c82edb55b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:44.598453 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f889e1cf-d55d-4773-b00f-257c82edb55b None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:44.620256 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 625/1240] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Jul 6 13:35:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:35:44.626621 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037102da-b2c3-4621-92d1-60a934596e45 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:44.747405 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037102da-b2c3-4621-92d1-60a934596e45 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:44.784464 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb7279b-13fc-4a3d-bb8c-5ddb19af8953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:44.785165 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7279b-13fc-4a3d-bb8c-5ddb19af8953 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:44.785448 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7279b-13fc-4a3d-bb8c-5ddb19af8953 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:44.785724 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7279b-13fc-4a3d-bb8c-5ddb19af8953 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:44.859820 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:44.866985 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb7279b-13fc-4a3d-bb8c-5ddb19af8953 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:44.868755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb7279b-13fc-4a3d-bb8c-5ddb19af8953 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:44.981988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 616/1241] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:35:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 13:35:44.988513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f889e1cf-d55d-4773-b00f-257c82edb55b None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:45.024493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:45.106831 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f889e1cf-d55d-4773-b00f-257c82edb55b None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:45.146452 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b6be0-1ddc-46bd-9e9a-e99777a72391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:45.147175 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b6be0-1ddc-46bd-9e9a-e99777a72391 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:45.147397 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b6be0-1ddc-46bd-9e9a-e99777a72391 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:45.147725 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b6be0-1ddc-46bd-9e9a-e99777a72391 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:45.158134 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b6be0-1ddc-46bd-9e9a-e99777a72391 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:45.159571 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b6be0-1ddc-46bd-9e9a-e99777a72391 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:45.177434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-133b6be0-1ddc-46bd-9e9a-e99777a72391 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:35:45.177434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:35:45.177434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:35:45.177434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:35:45.177434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:35:45.177434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:35:45.177434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:35:45.177434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:35:45.177434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:35:45.177434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:35:45.177434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:35:45.177434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:35:45.177434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:35:45.177434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:35:45.177434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:35:45.178578 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:35:45.178578 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:35:45.178578 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:35:45.178578 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:35:45.178578 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:35:45.178578 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:35:45.178578 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:35:45.178578 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:35:45.178578 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:35:45.178578 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:35:45.178578 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:35:45.178578 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:35:45.178578 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:35:45.178578 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:35:45.178578 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:35:45.178578 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:35:45.178578 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:35:45.180239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:35:45.180239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:35:45.180239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:35:45.180239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:35:45.180239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 626/1242] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:35:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:35:45.186459 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb7279b-13fc-4a3d-bb8c-5ddb19af8953 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:45.289991 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:45.303060 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb7279b-13fc-4a3d-bb8c-5ddb19af8953 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:45.346816 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad9ce6-85cc-4775-9b32-25957e0ca31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:45.347496 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad9ce6-85cc-4775-9b32-25957e0ca31a None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:45.347771 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad9ce6-85cc-4775-9b32-25957e0ca31a None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:45.348048 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad9ce6-85cc-4775-9b32-25957e0ca31a None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:45.348755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ad9ce6-85cc-4775-9b32-25957e0ca31a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:45.349149 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ad9ce6-85cc-4775-9b32-25957e0ca31a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:45.350485 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ad9ce6-85cc-4775-9b32-25957e0ca31a None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:45.363040 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 617/1243] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:35:45.370193 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b6be0-1ddc-46bd-9e9a-e99777a72391 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:45.462001 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:45.485400 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b6be0-1ddc-46bd-9e9a-e99777a72391 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:45.521802 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26936e8e-b9b1-4f3c-8de3-e9b305dcdcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:45.522621 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26936e8e-b9b1-4f3c-8de3-e9b305dcdcad None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:45.522964 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26936e8e-b9b1-4f3c-8de3-e9b305dcdcad None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:45.523307 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26936e8e-b9b1-4f3c-8de3-e9b305dcdcad None admin] PATH_INFO: `/v3/projects/service` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:45.530375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26936e8e-b9b1-4f3c-8de3-e9b305dcdcad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:45.532012 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26936e8e-b9b1-4f3c-8de3-e9b305dcdcad None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:45.546821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-26936e8e-b9b1-4f3c-8de3-e9b305dcdcad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:35:45.546821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:35:45.546821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:35:45.546821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:35:45.546821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:35:45.546821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:35:45.546821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:35:45.546821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:35:45.546821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:35:45.546821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:35:45.546821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:35:45.546821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:35:45.546821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:35:45.546821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:35:45.546821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:35:45.549381 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:35:45.549381 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:35:45.549381 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:35:45.549381 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:35:45.549381 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:35:45.549381 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:35:45.549381 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:35:45.549381 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:35:45.549381 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:35:45.549381 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:35:45.549381 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:35:45.549381 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:35:45.549381 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:35:45.549381 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:35:45.549381 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:35:45.549381 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:35:45.549381 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:35:45.550735 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:35:45.550735 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:35:45.550735 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:35:45.550735 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:35:45.550735 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:35:45.550735 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:35:45.550735 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:35:45.550735 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:35:45.550735 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 627/1244] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:35:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:35:45.559057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad9ce6-85cc-4775-9b32-25957e0ca31a None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:45.674588 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad9ce6-85cc-4775-9b32-25957e0ca31a None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:45.715261 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f388ce5-cae4-415a-8ab0-3554b4da95d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:45.715858 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f388ce5-cae4-415a-8ab0-3554b4da95d1 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:45.716145 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f388ce5-cae4-415a-8ab0-3554b4da95d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:45.716447 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f388ce5-cae4-415a-8ab0-3554b4da95d1 None admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:45.717417 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f388ce5-cae4-415a-8ab0-3554b4da95d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:45.717696 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f388ce5-cae4-415a-8ab0-3554b4da95d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:45.719208 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f388ce5-cae4-415a-8ab0-3554b4da95d1 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:45.731665 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 618/1245] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:35:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:35:45.737722 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26936e8e-b9b1-4f3c-8de3-e9b305dcdcad None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:45.856154 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26936e8e-b9b1-4f3c-8de3-e9b305dcdcad None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:45.889769 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a4ee7-c42a-405f-8b7c-14d53d656d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:45.890416 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a4ee7-c42a-405f-8b7c-14d53d656d82 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:45.890695 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a4ee7-c42a-405f-8b7c-14d53d656d82 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:45.890967 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a4ee7-c42a-405f-8b7c-14d53d656d82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:45.891788 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20a4ee7-c42a-405f-8b7c-14d53d656d82 None admin] RBAC: Adding query filter params (role.id=a6853d485f7942259b3646de3f386852, scope.project.id=bfe6c3d8da2743be9609184db2506303, user.id=890d2ae150d647a18dae25159cf97484) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:45.892096 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20a4ee7-c42a-405f-8b7c-14d53d656d82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:45.893545 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20a4ee7-c42a-405f-8b7c-14d53d656d82 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:45.899955 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 628/1246] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:35:45 2022] GET /identity/v3/role_assignments?user.id=890d2ae150d647a18dae25159cf97484&scope.project.id=bfe6c3d8da2743be9609184db2506303&role.id=a6853d485f7942259b3646de3f386852 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:35:48.116156 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ecbc8-9ae7-45ce-99b2-aeaad33ecb66 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:48.117464 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ecbc8-9ae7-45ce-99b2-aeaad33ecb66 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:48.117464 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ecbc8-9ae7-45ce-99b2-aeaad33ecb66 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:48.118822 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 619/1247] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:35:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:35:48.124227 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd9a4d-ce10-48c1-b5e0-600cdf1ddf7a None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:48.124663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd9a4d-ce10-48c1-b5e0-600cdf1ddf7a None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:48.124774 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd9a4d-ce10-48c1-b5e0-600cdf1ddf7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:48.130113 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:48.143707 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-a3dd9a4d-ce10-48c1-b5e0-600cdf1ddf7a None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:35:48.144891 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-a3dd9a4d-ce10-48c1-b5e0-600cdf1ddf7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:35:48.216935 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 629/1248] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:35:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 13:35:48.240750 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ecbc8-9ae7-45ce-99b2-aeaad33ecb66 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:48.262585 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:48.362512 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ecbc8-9ae7-45ce-99b2-aeaad33ecb66 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:48.400882 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbe7c68-813e-4192-bdfa-06f7082193d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:48.401536 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe7c68-813e-4192-bdfa-06f7082193d3 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:48.401812 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe7c68-813e-4192-bdfa-06f7082193d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:48.402159 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe7c68-813e-4192-bdfa-06f7082193d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:48.412527 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbe7c68-813e-4192-bdfa-06f7082193d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:35:48.421516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-adbe7c68-813e-4192-bdfa-06f7082193d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:35:48.421516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:35:48.421516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:35:48.421516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:35:48.421516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:35:48.421516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:35:48.421516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:35:48.421516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:35:48.421516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:35:48.421516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:35:48.421516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:35:48.421516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:35:48.421516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:35:48.421516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:35:48.421516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:35:48.421516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:35:48.424326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:35:48.424326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:35:48.424326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:35:48.424326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:35:48.424326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:35:48.424326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:35:48.424326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:35:48.424326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:35:48.424326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:35:48.424326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:35:48.424326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:35:48.424326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:35:48.424326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:35:48.424326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:35:48.424326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:35:48.424326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:35:48.426136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:35:48.426136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:35:48.426136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:35:48.426136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:35:48.426136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:35:48.426136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:35:48.426136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:35:48.426136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:35:48.426136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:35:48.426136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:35:48.426136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:35:48.426136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:35:48.426136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 620/1249] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:35:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:35:48.430377 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd9a4d-ce10-48c1-b5e0-600cdf1ddf7a None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:48.550715 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd9a4d-ce10-48c1-b5e0-600cdf1ddf7a None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:48.586575 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:48.593294 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f00a68-dca4-470e-a4f1-8d998bb0aa35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:48.594129 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f00a68-dca4-470e-a4f1-8d998bb0aa35 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:48.594482 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f00a68-dca4-470e-a4f1-8d998bb0aa35 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:48.594767 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f00a68-dca4-470e-a4f1-8d998bb0aa35 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:48.595564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f00a68-dca4-470e-a4f1-8d998bb0aa35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:48.595957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f00a68-dca4-470e-a4f1-8d998bb0aa35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:48.597648 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f00a68-dca4-470e-a4f1-8d998bb0aa35 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:48.607862 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 630/1250] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:35:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:35:48.614129 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbe7c68-813e-4192-bdfa-06f7082193d3 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:48.701214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:48.725353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbe7c68-813e-4192-bdfa-06f7082193d3 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:48.763314 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e4dc9-c2b7-4a6e-9c20-46e7254fe146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:48.764119 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e4dc9-c2b7-4a6e-9c20-46e7254fe146 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:48.764406 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e4dc9-c2b7-4a6e-9c20-46e7254fe146 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:48.764682 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e4dc9-c2b7-4a6e-9c20-46e7254fe146 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:48.775741 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e4dc9-c2b7-4a6e-9c20-46e7254fe146 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:48.777256 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e4dc9-c2b7-4a6e-9c20-46e7254fe146 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:48.794238 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-cd2e4dc9-c2b7-4a6e-9c20-46e7254fe146 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:35:48.794238 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:35:48.794238 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:35:48.794238 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:35:48.794238 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:35:48.794238 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:35:48.794238 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:35:48.794238 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:35:48.794238 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:35:48.794238 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:35:48.794238 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:35:48.794238 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:35:48.794238 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:35:48.794238 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:35:48.794238 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:35:48.795643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:35:48.795643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:35:48.795643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:35:48.795643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:35:48.795643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:35:48.795643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:35:48.795643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:35:48.795643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:35:48.795643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:35:48.795643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:35:48.795643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:35:48.795643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:35:48.795643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:35:48.795643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:35:48.795643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:35:48.795643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:35:48.795643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:35:48.796836 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:35:48.796836 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:35:48.796836 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:35:48.796836 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:35:48.796836 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 621/1251] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:35:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:35:48.801736 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f00a68-dca4-470e-a4f1-8d998bb0aa35 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:48.916561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f00a68-dca4-470e-a4f1-8d998bb0aa35 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:48.956044 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d698b-3542-4dcd-bb6a-4c823d6e4473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:48.956592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d698b-3542-4dcd-bb6a-4c823d6e4473 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:48.956872 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d698b-3542-4dcd-bb6a-4c823d6e4473 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:48.957178 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d698b-3542-4dcd-bb6a-4c823d6e4473 None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:48.957893 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d698b-3542-4dcd-bb6a-4c823d6e4473 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:48.958208 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d698b-3542-4dcd-bb6a-4c823d6e4473 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:48.959565 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d698b-3542-4dcd-bb6a-4c823d6e4473 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:48.975303 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 631/1252] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:35:48.982285 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e4dc9-c2b7-4a6e-9c20-46e7254fe146 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:49.102043 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e4dc9-c2b7-4a6e-9c20-46e7254fe146 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:49.130944 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:49.140245 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe7c58-0930-490c-8e72-1a62690aa2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:49.141937 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe7c58-0930-490c-8e72-1a62690aa2c1 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:49.142327 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe7c58-0930-490c-8e72-1a62690aa2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:49.142608 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe7c58-0930-490c-8e72-1a62690aa2c1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:49.152650 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe7c58-0930-490c-8e72-1a62690aa2c1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:49.155429 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe7c58-0930-490c-8e72-1a62690aa2c1 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:49.175953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-4afe7c58-0930-490c-8e72-1a62690aa2c1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 06 13:35:49.175953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:35:49.175953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:35:49.175953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:35:49.175953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:35:49.175953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:35:49.175953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:35:49.175953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:35:49.175953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:35:49.175953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:35:49.175953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:35:49.175953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:35:49.175953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:35:49.175953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:35:49.175953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:35:49.175953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:35:49.177101 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:35:49.177101 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:35:49.177101 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:35:49.177101 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:35:49.177101 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:35:49.177101 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:35:49.177101 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:35:49.177101 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:35:49.177101 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:35:49.177101 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:35:49.177101 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:35:49.177101 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:35:49.177101 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:35:49.177101 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:35:49.177101 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:35:49.177101 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:35:49.177101 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:35:49.177101 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:35:49.183171 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:35:49.183171 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:35:49.183171 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:35:49.183171 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:35:49.183171 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:35:49.183171 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:35:49.183171 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:35:49.183171 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:35:49.183171 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 06 13:35:49.183171 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:35:49.183171 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 622/1253] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:35:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:35:49.184872 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d698b-3542-4dcd-bb6a-4c823d6e4473 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:49.250952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:49.301368 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d698b-3542-4dcd-bb6a-4c823d6e4473 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:49.339331 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39a354-1d12-4b01-bb87-bb50c53b6c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:49.340001 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39a354-1d12-4b01-bb87-bb50c53b6c7b None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:49.340281 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39a354-1d12-4b01-bb87-bb50c53b6c7b None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:49.340563 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39a354-1d12-4b01-bb87-bb50c53b6c7b None admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:49.341435 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39a354-1d12-4b01-bb87-bb50c53b6c7b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:49.341749 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39a354-1d12-4b01-bb87-bb50c53b6c7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:49.343175 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39a354-1d12-4b01-bb87-bb50c53b6c7b None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:49.361525 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 632/1254] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Jul 6 13:35:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:35:49.368140 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe7c58-0930-490c-8e72-1a62690aa2c1 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:49.482395 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe7c58-0930-490c-8e72-1a62690aa2c1 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:49.531996 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d410b20-454d-40cc-a210-68be7df47700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:49.532570 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d410b20-454d-40cc-a210-68be7df47700 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:49.532866 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d410b20-454d-40cc-a210-68be7df47700 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:49.533168 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d410b20-454d-40cc-a210-68be7df47700 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:49.598085 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:49.611949 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d410b20-454d-40cc-a210-68be7df47700 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:49.613434 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d410b20-454d-40cc-a210-68be7df47700 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:49.719741 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 623/1255] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:35:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 06 13:35:49.725814 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39a354-1d12-4b01-bb87-bb50c53b6c7b None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:49.839937 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39a354-1d12-4b01-bb87-bb50c53b6c7b None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:49.854450 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:49.878248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520a61b-d2af-4f2e-a280-02b21ae77635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:49.879154 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520a61b-d2af-4f2e-a280-02b21ae77635 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:49.879473 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520a61b-d2af-4f2e-a280-02b21ae77635 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:49.879747 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520a61b-d2af-4f2e-a280-02b21ae77635 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:49.891479 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520a61b-d2af-4f2e-a280-02b21ae77635 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:49.893075 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520a61b-d2af-4f2e-a280-02b21ae77635 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:49.912729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-2520a61b-d2af-4f2e-a280-02b21ae77635 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:35:49.912729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:35:49.912729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:35:49.912729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:35:49.912729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:35:49.912729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:35:49.912729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:35:49.912729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:35:49.912729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:35:49.912729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:35:49.912729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:35:49.912729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:35:49.912729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:35:49.912729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:35:49.912729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:35:49.913966 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:35:49.913966 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:35:49.913966 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:35:49.913966 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:35:49.913966 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:35:49.913966 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:35:49.913966 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:35:49.913966 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:35:49.913966 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:35:49.913966 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:35:49.913966 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:35:49.913966 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:35:49.913966 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:35:49.913966 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:35:49.913966 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:35:49.913966 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:35:49.913966 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:35:49.915120 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:35:49.915120 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:35:49.915120 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:35:49.915120 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:35:49.915120 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 633/1256] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:35:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:35:49.920551 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d410b20-454d-40cc-a210-68be7df47700 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:50.034196 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:50.048259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d410b20-454d-40cc-a210-68be7df47700 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:50.088473 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aacf4d2-acea-4e17-91ad-2747e01268b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:50.089171 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aacf4d2-acea-4e17-91ad-2747e01268b2 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:50.089457 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aacf4d2-acea-4e17-91ad-2747e01268b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:50.089729 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aacf4d2-acea-4e17-91ad-2747e01268b2 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:50.090493 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aacf4d2-acea-4e17-91ad-2747e01268b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:50.090747 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aacf4d2-acea-4e17-91ad-2747e01268b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:50.092104 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aacf4d2-acea-4e17-91ad-2747e01268b2 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:50.108962 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 624/1257] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:35:50.110346 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520a61b-d2af-4f2e-a280-02b21ae77635 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:50.213708 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520a61b-d2af-4f2e-a280-02b21ae77635 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:50.251772 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7264aee-89c6-419f-a4bc-367a91849e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:50.252682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7264aee-89c6-419f-a4bc-367a91849e3e None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:50.253025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7264aee-89c6-419f-a4bc-367a91849e3e None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:50.253371 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7264aee-89c6-419f-a4bc-367a91849e3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:50.261275 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7264aee-89c6-419f-a4bc-367a91849e3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:50.262725 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7264aee-89c6-419f-a4bc-367a91849e3e None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:50.277336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-c7264aee-89c6-419f-a4bc-367a91849e3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:35:50.277336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:35:50.277336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:35:50.277336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:35:50.277336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:35:50.277336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:35:50.277336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:35:50.277336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:35:50.277336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:35:50.277336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:35:50.277336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:35:50.277336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:35:50.277336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:35:50.277336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:35:50.277336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:35:50.278348 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:35:50.278348 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:35:50.278348 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:35:50.278348 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:35:50.278348 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:35:50.278348 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:35:50.278348 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:35:50.278348 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:35:50.278348 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:35:50.278348 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:35:50.278348 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:35:50.278348 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:35:50.278348 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:35:50.278348 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:35:50.278348 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:35:50.278348 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:35:50.278348 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:35:50.279540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:35:50.279540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:35:50.279540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:35:50.279540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:35:50.279540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:35:50.279540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:35:50.279540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:35:50.279540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:35:50.279540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 634/1258] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:35:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:35:50.286167 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aacf4d2-acea-4e17-91ad-2747e01268b2 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:50.397456 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aacf4d2-acea-4e17-91ad-2747e01268b2 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:50.434819 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6973a08-880c-43b2-a1f8-7afeaf63451d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:50.435484 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6973a08-880c-43b2-a1f8-7afeaf63451d None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:50.435766 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6973a08-880c-43b2-a1f8-7afeaf63451d None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:50.436039 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6973a08-880c-43b2-a1f8-7afeaf63451d None admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:50.436821 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6973a08-880c-43b2-a1f8-7afeaf63451d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:50.437163 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6973a08-880c-43b2-a1f8-7afeaf63451d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:50.438577 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6973a08-880c-43b2-a1f8-7afeaf63451d None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:50.453524 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 625/1259] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:35:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:35:50.459996 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7264aee-89c6-419f-a4bc-367a91849e3e None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:50.488184 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:50.573296 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7264aee-89c6-419f-a4bc-367a91849e3e None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:50.610324 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d8718-2660-4c0d-896b-01975bbcffb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:50.611064 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d8718-2660-4c0d-896b-01975bbcffb1 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:50.611359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d8718-2660-4c0d-896b-01975bbcffb1 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:50.611753 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d8718-2660-4c0d-896b-01975bbcffb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:50.612680 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d8718-2660-4c0d-896b-01975bbcffb1 None admin] RBAC: Adding query filter params (role.id=795daa2c78354fd48e4a741e1182cc98, scope.project.id=bfe6c3d8da2743be9609184db2506303, user.id=890d2ae150d647a18dae25159cf97484) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:50.613072 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d8718-2660-4c0d-896b-01975bbcffb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:50.614685 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d8718-2660-4c0d-896b-01975bbcffb1 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:50.621091 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 635/1260] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:35:50 2022] GET /identity/v3/role_assignments?user.id=890d2ae150d647a18dae25159cf97484&scope.project.id=bfe6c3d8da2743be9609184db2506303&role.id=795daa2c78354fd48e4a741e1182cc98 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:35:52.680569 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef42f1-7516-45e7-9b1d-4e40223cd929 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:52.681098 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef42f1-7516-45e7-9b1d-4e40223cd929 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:52.681098 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef42f1-7516-45e7-9b1d-4e40223cd929 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:52.682545 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 626/1261] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:35:52.687807 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f541dc-fc30-4c44-881b-729bddcc8520 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:52.688221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f541dc-fc30-4c44-881b-729bddcc8520 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:52.688369 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f541dc-fc30-4c44-881b-729bddcc8520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:52.705573 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-13f541dc-fc30-4c44-881b-729bddcc8520 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:35:52.706749 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-13f541dc-fc30-4c44-881b-729bddcc8520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:35:52.771688 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 636/1262] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:35:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:35:52.800621 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef42f1-7516-45e7-9b1d-4e40223cd929 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:52.811325 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:52.899455 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef42f1-7516-45e7-9b1d-4e40223cd929 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:52.931681 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b492f0-f8ac-4b21-a0a8-f4bdee59a5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:52.932384 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b492f0-f8ac-4b21-a0a8-f4bdee59a5ee None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:52.932665 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b492f0-f8ac-4b21-a0a8-f4bdee59a5ee None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:52.932936 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b492f0-f8ac-4b21-a0a8-f4bdee59a5ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:52.942762 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b492f0-f8ac-4b21-a0a8-f4bdee59a5ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:35:52.950935 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-69b492f0-f8ac-4b21-a0a8-f4bdee59a5ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:35:52.950935 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:35:52.950935 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:35:52.950935 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:35:52.950935 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:35:52.950935 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:35:52.950935 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:35:52.950935 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:35:52.950935 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:35:52.950935 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:35:52.950935 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:35:52.950935 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:35:52.950935 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:35:52.950935 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:35:52.950935 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:35:52.950935 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:35:52.952052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:35:52.952052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:35:52.952052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:35:52.952052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:35:52.952052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:35:52.952052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:35:52.952052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:35:52.952052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:35:52.952052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:35:52.952052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:35:52.952052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:35:52.952052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:35:52.952052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:35:52.952052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:35:52.952052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:35:52.952052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:35:52.953259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:35:52.953259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:35:52.953259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:35:52.953259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:35:52.953259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:35:52.953259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:35:52.953259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:35:52.953259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:35:52.953259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:35:52.953259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:35:52.953259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:35:52.953259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:35:52.953259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 627/1263] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:35:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:35:52.958794 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f541dc-fc30-4c44-881b-729bddcc8520 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:52.980885 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:53.064007 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f541dc-fc30-4c44-881b-729bddcc8520 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:53.097107 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216efe12-dbe9-4a18-a903-02c087981b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:53.097764 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216efe12-dbe9-4a18-a903-02c087981b56 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:53.098048 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216efe12-dbe9-4a18-a903-02c087981b56 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:53.098324 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216efe12-dbe9-4a18-a903-02c087981b56 None admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:53.099066 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216efe12-dbe9-4a18-a903-02c087981b56 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:53.099374 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216efe12-dbe9-4a18-a903-02c087981b56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:53.100727 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216efe12-dbe9-4a18-a903-02c087981b56 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:53.108722 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 637/1264] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:35:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:35:53.114465 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b492f0-f8ac-4b21-a0a8-f4bdee59a5ee None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:53.178393 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:53.211666 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b492f0-f8ac-4b21-a0a8-f4bdee59a5ee None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:53.243266 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bdef0-2c64-42ce-ace5-a60c369aaaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:53.244040 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bdef0-2c64-42ce-ace5-a60c369aaaec None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:53.244322 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bdef0-2c64-42ce-ace5-a60c369aaaec None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:53.244597 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bdef0-2c64-42ce-ace5-a60c369aaaec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:53.254670 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6bdef0-2c64-42ce-ace5-a60c369aaaec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:53.256086 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6bdef0-2c64-42ce-ace5-a60c369aaaec None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:53.273316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-9a6bdef0-2c64-42ce-ace5-a60c369aaaec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:35:53.273316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:35:53.273316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:35:53.273316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:35:53.273316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:35:53.273316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:35:53.273316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:35:53.273316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:35:53.273316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:35:53.273316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:35:53.273316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:35:53.273316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:35:53.273316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:35:53.273316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:35:53.273316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:35:53.274338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:35:53.274338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:35:53.274338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:35:53.274338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:35:53.274338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:35:53.274338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:35:53.274338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:35:53.274338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:35:53.274338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:35:53.274338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:35:53.274338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:35:53.274338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:35:53.274338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:35:53.274338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:35:53.274338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:35:53.274338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:35:53.274338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:35:53.275577 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:35:53.275577 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:35:53.275577 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:35:53.275577 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:35:53.275577 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 628/1265] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:35:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:35:53.280305 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216efe12-dbe9-4a18-a903-02c087981b56 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:53.361120 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:53.380755 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216efe12-dbe9-4a18-a903-02c087981b56 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:53.412823 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71810b-4c85-48c7-941a-85c4f8bbd0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:53.413556 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71810b-4c85-48c7-941a-85c4f8bbd0f4 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:53.413842 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71810b-4c85-48c7-941a-85c4f8bbd0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:53.414071 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71810b-4c85-48c7-941a-85c4f8bbd0f4 None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:53.414784 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb71810b-4c85-48c7-941a-85c4f8bbd0f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:53.415083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb71810b-4c85-48c7-941a-85c4f8bbd0f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:53.416421 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb71810b-4c85-48c7-941a-85c4f8bbd0f4 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:53.427769 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 638/1266] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:35:53.434007 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bdef0-2c64-42ce-ace5-a60c369aaaec None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:53.530918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bdef0-2c64-42ce-ace5-a60c369aaaec None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:53.550173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:53.564417 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69344c4-6c96-4aa3-af75-9a69a8f22052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:53.565232 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69344c4-6c96-4aa3-af75-9a69a8f22052 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:53.565522 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69344c4-6c96-4aa3-af75-9a69a8f22052 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:53.565803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69344c4-6c96-4aa3-af75-9a69a8f22052 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:53.572985 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69344c4-6c96-4aa3-af75-9a69a8f22052 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:53.574442 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69344c4-6c96-4aa3-af75-9a69a8f22052 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:53.588369 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-c69344c4-6c96-4aa3-af75-9a69a8f22052 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 06 13:35:53.588369 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:35:53.588369 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:35:53.588369 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:35:53.588369 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:35:53.588369 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:35:53.588369 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:35:53.588369 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:35:53.588369 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:35:53.588369 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:35:53.588369 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:35:53.588369 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:35:53.588369 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:35:53.588369 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:35:53.588369 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:35:53.588369 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:35:53.589541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:35:53.589541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:35:53.589541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:35:53.589541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:35:53.589541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:35:53.589541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:35:53.589541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:35:53.589541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:35:53.589541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:35:53.589541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:35:53.589541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:35:53.589541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:35:53.589541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:35:53.589541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:35:53.589541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:35:53.589541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:35:53.589541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:35:53.589541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:35:53.590536 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:35:53.590536 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:35:53.590536 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:35:53.590536 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:35:53.590536 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:35:53.590536 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:35:53.590536 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:35:53.590536 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:35:53.590536 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 06 13:35:53.590536 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:35:53.590536 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 629/1267] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:35:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:35:53.595323 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71810b-4c85-48c7-941a-85c4f8bbd0f4 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:53.692120 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71810b-4c85-48c7-941a-85c4f8bbd0f4 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:53.725364 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c7021-1d4a-43e6-a0c8-313969a829e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:53.726328 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c7021-1d4a-43e6-a0c8-313969a829e6 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:53.726328 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c7021-1d4a-43e6-a0c8-313969a829e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:53.726603 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c7021-1d4a-43e6-a0c8-313969a829e6 None admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:53.727441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c7021-1d4a-43e6-a0c8-313969a829e6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:53.727752 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c7021-1d4a-43e6-a0c8-313969a829e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:53.729199 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c7021-1d4a-43e6-a0c8-313969a829e6 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:53.737508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:53.746180 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 639/1268] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Jul 6 13:35:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:35:53.751845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69344c4-6c96-4aa3-af75-9a69a8f22052 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:53.856099 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69344c4-6c96-4aa3-af75-9a69a8f22052 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:53.888476 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01716430-17ca-4241-b64c-b37b94a134dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:53.889285 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01716430-17ca-4241-b64c-b37b94a134dc None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:53.889573 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01716430-17ca-4241-b64c-b37b94a134dc None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:53.889851 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01716430-17ca-4241-b64c-b37b94a134dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:53.900604 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01716430-17ca-4241-b64c-b37b94a134dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:53.902066 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01716430-17ca-4241-b64c-b37b94a134dc None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:53.919113 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-01716430-17ca-4241-b64c-b37b94a134dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:35:53.919113 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:35:53.919113 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:35:53.919113 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:35:53.919113 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:35:53.919113 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:35:53.919113 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:35:53.919113 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:35:53.919113 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:35:53.919113 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:35:53.919113 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:35:53.919113 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:35:53.919113 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:35:53.919113 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:35:53.919113 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:35:53.920141 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:35:53.920141 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:35:53.920141 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:35:53.920141 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:35:53.920141 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:35:53.920141 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:35:53.920141 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:35:53.920141 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:35:53.920141 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:35:53.920141 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:35:53.920141 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:35:53.920141 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:35:53.920141 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:35:53.920141 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:35:53.920141 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:35:53.920141 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:35:53.920141 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:35:53.921282 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:35:53.921282 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:35:53.921282 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:35:53.921282 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:35:53.921282 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 630/1269] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:35:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:35:53.926399 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c7021-1d4a-43e6-a0c8-313969a829e6 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:54.027130 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c7021-1d4a-43e6-a0c8-313969a829e6 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:54.059930 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f938df-0669-4c7a-bd9d-b7b07ba0f640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:54.060577 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f938df-0669-4c7a-bd9d-b7b07ba0f640 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:54.060860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f938df-0669-4c7a-bd9d-b7b07ba0f640 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:54.061168 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f938df-0669-4c7a-bd9d-b7b07ba0f640 None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:54.061894 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f938df-0669-4c7a-bd9d-b7b07ba0f640 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:54.062199 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f938df-0669-4c7a-bd9d-b7b07ba0f640 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:54.063553 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f938df-0669-4c7a-bd9d-b7b07ba0f640 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:54.074410 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 640/1270] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:35:54.080123 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01716430-17ca-4241-b64c-b37b94a134dc None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:54.111866 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:54.179846 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01716430-17ca-4241-b64c-b37b94a134dc None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:54.213089 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab8e6d-d53a-4f31-a1a6-090811c644f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:54.213827 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab8e6d-d53a-4f31-a1a6-090811c644f1 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:54.214113 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab8e6d-d53a-4f31-a1a6-090811c644f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:54.214415 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab8e6d-d53a-4f31-a1a6-090811c644f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:54.220838 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab8e6d-d53a-4f31-a1a6-090811c644f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:54.222301 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab8e6d-d53a-4f31-a1a6-090811c644f1 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:54.237438 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-aaab8e6d-d53a-4f31-a1a6-090811c644f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:35:54.237438 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:35:54.237438 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:35:54.237438 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:35:54.237438 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:35:54.237438 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:35:54.237438 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:35:54.237438 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:35:54.237438 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:35:54.237438 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:35:54.237438 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:35:54.237438 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:35:54.237438 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:35:54.237438 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:35:54.237438 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:35:54.238490 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:35:54.238490 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:35:54.238490 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:35:54.238490 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:35:54.238490 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:35:54.238490 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:35:54.238490 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:35:54.238490 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:35:54.238490 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:35:54.238490 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:35:54.238490 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:35:54.238490 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:35:54.238490 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:35:54.238490 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:35:54.238490 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:35:54.238490 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:35:54.238490 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:35:54.239590 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:35:54.239590 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:35:54.239590 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:35:54.239590 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:35:54.239590 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:35:54.239590 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:35:54.239590 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:35:54.239590 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:35:54.239590 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 631/1271] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:35:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:35:54.244476 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f938df-0669-4c7a-bd9d-b7b07ba0f640 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:54.302020 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:54.343975 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f938df-0669-4c7a-bd9d-b7b07ba0f640 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:54.375937 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394201d8-893d-4390-8007-a2146195df10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:54.376527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394201d8-893d-4390-8007-a2146195df10 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:54.376812 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394201d8-893d-4390-8007-a2146195df10 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:54.377119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394201d8-893d-4390-8007-a2146195df10 None admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:54.377907 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394201d8-893d-4390-8007-a2146195df10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:54.378216 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394201d8-893d-4390-8007-a2146195df10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:54.379634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394201d8-893d-4390-8007-a2146195df10 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:54.391796 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 641/1272] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:35:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:35:54.397710 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab8e6d-d53a-4f31-a1a6-090811c644f1 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:54.488401 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:54.497602 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab8e6d-d53a-4f31-a1a6-090811c644f1 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:54.530484 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50732177-09a8-4a47-8d3c-0bde51826a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:54.531359 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50732177-09a8-4a47-8d3c-0bde51826a2d None admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:54.531637 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50732177-09a8-4a47-8d3c-0bde51826a2d None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:54.531909 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50732177-09a8-4a47-8d3c-0bde51826a2d None admin] PATH_INFO: `/v3/projects/bfe6c3d8da2743be9609184db2506303/users/890d2ae150d647a18dae25159cf97484/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:54.533834 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50732177-09a8-4a47-8d3c-0bde51826a2d None admin] RBAC: Authorizing `identity:create_grant(project_id=bfe6c3d8da2743be9609184db2506303, user_id=890d2ae150d647a18dae25159cf97484, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:54.535250 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50732177-09a8-4a47-8d3c-0bde51826a2d None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:54.547225 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 632/1273] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jul 6 13:35:54 2022] PUT /identity/v3/projects/bfe6c3d8da2743be9609184db2506303/users/890d2ae150d647a18dae25159cf97484/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:35:56.536392 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce5fee-2e93-4468-8cfb-6c17c6f43fc2 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:56.537053 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce5fee-2e93-4468-8cfb-6c17c6f43fc2 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:56.537053 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce5fee-2e93-4468-8cfb-6c17c6f43fc2 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:56.538368 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 642/1274] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:35:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:35:56.544017 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a4ab7-0e21-48e2-9491-be6b962c41d0 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:56.544497 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a4ab7-0e21-48e2-9491-be6b962c41d0 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:56.544607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a4ab7-0e21-48e2-9491-be6b962c41d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:56.561425 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-4c8a4ab7-0e21-48e2-9491-be6b962c41d0 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:35:56.562605 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-4c8a4ab7-0e21-48e2-9491-be6b962c41d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:35:56.627121 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 633/1275] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:35:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:35:56.651908 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce5fee-2e93-4468-8cfb-6c17c6f43fc2 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:56.750462 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce5fee-2e93-4468-8cfb-6c17c6f43fc2 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:56.766944 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:56.783055 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5785b-e53c-4dc4-8a0d-69bd40eaa71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:56.783824 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5785b-e53c-4dc4-8a0d-69bd40eaa71a None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:56.784100 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5785b-e53c-4dc4-8a0d-69bd40eaa71a None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:56.784368 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5785b-e53c-4dc4-8a0d-69bd40eaa71a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:56.794896 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d5785b-e53c-4dc4-8a0d-69bd40eaa71a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 13:35:56.802675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-45d5785b-e53c-4dc4-8a0d-69bd40eaa71a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:35:56.802675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:35:56.802675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:35:56.802675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:35:56.802675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:35:56.802675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:35:56.802675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:35:56.802675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:35:56.802675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:35:56.802675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:35:56.802675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:35:56.802675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:35:56.802675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 13:35:56.802675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 13:35:56.802675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 13:35:56.802675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 13:35:56.803851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 13:35:56.803851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 13:35:56.803851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 13:35:56.803851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 13:35:56.803851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:35:56.803851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:35:56.803851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:35:56.803851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:35:56.803851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:35:56.803851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:35:56.803851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:35:56.803851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:35:56.803851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:35:56.803851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:35:56.803851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:35:56.803851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:35:56.805745 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:35:56.805745 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:35:56.805745 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:35:56.805745 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:35:56.805745 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 13:35:56.805745 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 13:35:56.805745 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 13:35:56.805745 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 13:35:56.805745 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 13:35:56.805745 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 13:35:56.805745 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 13:35:56.805745 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:35:56.805745 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 643/1276] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jul 6 13:35:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:35:56.810383 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a4ab7-0e21-48e2-9491-be6b962c41d0 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:56.887220 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:56.905784 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a4ab7-0e21-48e2-9491-be6b962c41d0 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:56.939143 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135d99e-f08b-457b-8d86-f733b70b6244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:56.939792 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135d99e-f08b-457b-8d86-f733b70b6244 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:56.940073 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135d99e-f08b-457b-8d86-f733b70b6244 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:56.940347 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135d99e-f08b-457b-8d86-f733b70b6244 None admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:56.941109 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3135d99e-f08b-457b-8d86-f733b70b6244 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:56.941412 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3135d99e-f08b-457b-8d86-f733b70b6244 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:56.942762 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3135d99e-f08b-457b-8d86-f733b70b6244 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:56.950877 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 634/1277] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jul 6 13:35:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:35:56.957157 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5785b-e53c-4dc4-8a0d-69bd40eaa71a None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:57.061237 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5785b-e53c-4dc4-8a0d-69bd40eaa71a None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:57.094308 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e16fac-03f3-481b-ae21-e985e55b14e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:57.095090 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e16fac-03f3-481b-ae21-e985e55b14e8 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:57.095375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e16fac-03f3-481b-ae21-e985e55b14e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:57.095654 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e16fac-03f3-481b-ae21-e985e55b14e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:57.106545 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e16fac-03f3-481b-ae21-e985e55b14e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:57.107984 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e16fac-03f3-481b-ae21-e985e55b14e8 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:57.124587 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-46e16fac-03f3-481b-ae21-e985e55b14e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:35:57.124587 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:35:57.124587 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:35:57.124587 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:35:57.124587 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:35:57.124587 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:35:57.124587 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:35:57.124587 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:35:57.124587 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:35:57.124587 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:35:57.124587 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:35:57.124587 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:35:57.124587 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:35:57.124587 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:35:57.124587 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:35:57.125659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:35:57.125659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:35:57.125659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:35:57.125659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:35:57.125659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:35:57.125659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:35:57.125659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:35:57.125659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:35:57.125659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:35:57.125659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:35:57.125659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:35:57.125659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:35:57.125659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:35:57.125659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:35:57.125659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:35:57.125659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:35:57.125659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:35:57.126979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:35:57.126979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:35:57.126979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:35:57.126979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:35:57.126979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 644/1278] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:35:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:35:57.131505 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135d99e-f08b-457b-8d86-f733b70b6244 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:57.228558 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135d99e-f08b-457b-8d86-f733b70b6244 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:57.259886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de55a4cf-21c6-4393-ad6f-020153eb7358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:57.260532 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55a4cf-21c6-4393-ad6f-020153eb7358 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:57.260811 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55a4cf-21c6-4393-ad6f-020153eb7358 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:57.261111 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55a4cf-21c6-4393-ad6f-020153eb7358 None admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:57.261817 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de55a4cf-21c6-4393-ad6f-020153eb7358 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:57.262121 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de55a4cf-21c6-4393-ad6f-020153eb7358 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:57.263464 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de55a4cf-21c6-4393-ad6f-020153eb7358 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:57.275479 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 635/1279] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:35:57.281111 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e16fac-03f3-481b-ae21-e985e55b14e8 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:57.312506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:57.380394 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e16fac-03f3-481b-ae21-e985e55b14e8 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:57.414841 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8f909-ac88-4a3d-9e67-5f2362f0ff5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:57.415624 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8f909-ac88-4a3d-9e67-5f2362f0ff5a None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:57.415910 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8f909-ac88-4a3d-9e67-5f2362f0ff5a None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:57.416189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8f909-ac88-4a3d-9e67-5f2362f0ff5a None admin] PATH_INFO: `/v3/users/placement` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:57.423563 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8f909-ac88-4a3d-9e67-5f2362f0ff5a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:57.424979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8f909-ac88-4a3d-9e67-5f2362f0ff5a None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:57.439869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-4bc8f909-ac88-4a3d-9e67-5f2362f0ff5a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 06 13:35:57.439869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:35:57.439869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:35:57.439869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:35:57.439869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:35:57.439869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:35:57.439869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:35:57.439869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:35:57.439869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:35:57.439869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:35:57.439869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:35:57.439869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:35:57.439869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 13:35:57.439869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 13:35:57.439869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 13:35:57.439869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 13:35:57.440981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:35:57.440981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:35:57.440981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 13:35:57.440981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:35:57.440981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 13:35:57.440981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 13:35:57.440981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:35:57.440981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:35:57.440981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:35:57.440981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:35:57.440981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:35:57.440981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:35:57.440981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:35:57.440981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:35:57.440981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:35:57.440981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:35:57.440981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:35:57.440981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:35:57.442014 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:35:57.442014 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:35:57.442014 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 13:35:57.442014 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 13:35:57.442014 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 13:35:57.442014 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 13:35:57.442014 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 13:35:57.442014 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 13:35:57.442014 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 06 13:35:57.442014 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:35:57.442014 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 645/1280] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:35:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:35:57.447306 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de55a4cf-21c6-4393-ad6f-020153eb7358 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:57.451024 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:57.544513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de55a4cf-21c6-4393-ad6f-020153eb7358 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:57.578842 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7522a72-1e9f-4e23-a178-0a821b8a3c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:57.579441 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7522a72-1e9f-4e23-a178-0a821b8a3c54 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:57.579723 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7522a72-1e9f-4e23-a178-0a821b8a3c54 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:57.579998 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7522a72-1e9f-4e23-a178-0a821b8a3c54 None admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:57.580828 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7522a72-1e9f-4e23-a178-0a821b8a3c54 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:57.581169 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7522a72-1e9f-4e23-a178-0a821b8a3c54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:57.582588 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7522a72-1e9f-4e23-a178-0a821b8a3c54 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:57.598936 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 636/1281] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Jul 6 13:35:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:35:57.605082 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8f909-ac88-4a3d-9e67-5f2362f0ff5a None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:57.691808 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:57.701429 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8f909-ac88-4a3d-9e67-5f2362f0ff5a None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:57.733702 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4800d7e-8526-4f8a-8502-fade9784b37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:57.734328 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4800d7e-8526-4f8a-8502-fade9784b37f None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:57.734608 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4800d7e-8526-4f8a-8502-fade9784b37f None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:57.734884 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4800d7e-8526-4f8a-8502-fade9784b37f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:57.798664 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4800d7e-8526-4f8a-8502-fade9784b37f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:57.800077 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4800d7e-8526-4f8a-8502-fade9784b37f None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:57.883713 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:57.893750 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 646/1282] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jul 6 13:35:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 13:35:57.899499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7522a72-1e9f-4e23-a178-0a821b8a3c54 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:57.960677 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:58.001134 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7522a72-1e9f-4e23-a178-0a821b8a3c54 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:58.034361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04e8b5-838d-4efc-b66c-5625e5bc94ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:58.035150 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04e8b5-838d-4efc-b66c-5625e5bc94ed None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:58.035438 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04e8b5-838d-4efc-b66c-5625e5bc94ed None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:58.035717 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04e8b5-838d-4efc-b66c-5625e5bc94ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:58.046086 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04e8b5-838d-4efc-b66c-5625e5bc94ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:58.047769 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04e8b5-838d-4efc-b66c-5625e5bc94ed None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:58.066449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-eb04e8b5-838d-4efc-b66c-5625e5bc94ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:35:58.066449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:35:58.066449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:35:58.066449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:35:58.066449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:35:58.066449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:35:58.066449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:35:58.066449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:35:58.066449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:35:58.066449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:35:58.066449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:35:58.066449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:35:58.066449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 13:35:58.066449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 13:35:58.066449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 13:35:58.067502 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 13:35:58.067502 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:35:58.067502 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:35:58.067502 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:35:58.067502 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:35:58.067502 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:35:58.067502 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:35:58.067502 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:35:58.067502 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:35:58.067502 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:35:58.067502 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:35:58.067502 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:35:58.067502 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:35:58.067502 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:35:58.067502 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:35:58.067502 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:35:58.067502 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:35:58.068718 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 13:35:58.068718 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 13:35:58.068718 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 13:35:58.068718 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:35:58.068718 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 637/1283] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:35:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:35:58.073466 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4800d7e-8526-4f8a-8502-fade9784b37f None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:58.172591 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4800d7e-8526-4f8a-8502-fade9784b37f None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:58.205977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413acf1d-ff65-454f-8c1a-2f067ca65b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:58.206659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413acf1d-ff65-454f-8c1a-2f067ca65b8b None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:58.206942 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413acf1d-ff65-454f-8c1a-2f067ca65b8b None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:58.207219 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413acf1d-ff65-454f-8c1a-2f067ca65b8b None admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:58.207937 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413acf1d-ff65-454f-8c1a-2f067ca65b8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:58.208246 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413acf1d-ff65-454f-8c1a-2f067ca65b8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:58.209633 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413acf1d-ff65-454f-8c1a-2f067ca65b8b None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:58.221075 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 647/1284] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jul 6 13:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:35:58.226744 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04e8b5-838d-4efc-b66c-5625e5bc94ed None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:58.323668 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04e8b5-838d-4efc-b66c-5625e5bc94ed None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:58.337020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:58.356860 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c792f-758d-421d-8b3e-903e8df713e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:58.357695 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c792f-758d-421d-8b3e-903e8df713e4 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:58.357975 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c792f-758d-421d-8b3e-903e8df713e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:58.358248 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c792f-758d-421d-8b3e-903e8df713e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:58.364587 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c792f-758d-421d-8b3e-903e8df713e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:58.366049 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c792f-758d-421d-8b3e-903e8df713e4 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:58.379843 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application [None req-b45c792f-758d-421d-8b3e-903e8df713e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:35:58.379843 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:35:58.379843 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:35:58.379843 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:35:58.379843 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:35:58.379843 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:35:58.379843 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:35:58.379843 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:35:58.379843 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:35:58.379843 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:35:58.379843 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:35:58.379843 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:35:58.379843 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 13:35:58.379843 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 13:35:58.379843 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 13:35:58.380883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 13:35:58.380883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:35:58.380883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:35:58.380883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:35:58.380883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:35:58.380883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:35:58.380883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:35:58.380883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:35:58.380883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application with Lock( Jul 06 13:35:58.380883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:35:58.380883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:35:58.380883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:35:58.380883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:35:58.380883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:35:58.380883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:35:58.380883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:35:58.380883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:35:58.382088 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 13:35:58.382088 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 13:35:58.382088 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 13:35:58.382088 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 13:35:58.382088 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 13:35:58.382088 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 13:35:58.382088 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 13:35:58.382088 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: ERROR keystone.server.flask.application Jul 06 13:35:58.382088 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 638/1285] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jul 6 13:35:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:35:58.386865 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413acf1d-ff65-454f-8c1a-2f067ca65b8b None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:58.435224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:35:58.484112 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413acf1d-ff65-454f-8c1a-2f067ca65b8b None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:58.515635 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6726c-b484-4d5f-9284-59b8c944af78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:58.516279 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6726c-b484-4d5f-9284-59b8c944af78 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:58.516560 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6726c-b484-4d5f-9284-59b8c944af78 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:58.516834 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6726c-b484-4d5f-9284-59b8c944af78 None admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:58.517643 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec6726c-b484-4d5f-9284-59b8c944af78 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:58.517953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec6726c-b484-4d5f-9284-59b8c944af78 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:58.519372 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec6726c-b484-4d5f-9284-59b8c944af78 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:58.531487 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 648/1286] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jul 6 13:35:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:35:58.537397 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c792f-758d-421d-8b3e-903e8df713e4 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:35:58.634887 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c792f-758d-421d-8b3e-903e8df713e4 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:35:58.667942 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29778b20-8ecd-4310-8baa-9b7f7ad72efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:35:58.668581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29778b20-8ecd-4310-8baa-9b7f7ad72efa None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:35:58.668861 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29778b20-8ecd-4310-8baa-9b7f7ad72efa None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:35:58.669162 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29778b20-8ecd-4310-8baa-9b7f7ad72efa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:35:58.669982 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29778b20-8ecd-4310-8baa-9b7f7ad72efa None admin] RBAC: Adding query filter params (role.id=795daa2c78354fd48e4a741e1182cc98, scope.project.id=bfe6c3d8da2743be9609184db2506303, user.id=890d2ae150d647a18dae25159cf97484) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:35:58.670347 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29778b20-8ecd-4310-8baa-9b7f7ad72efa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:35:58.671798 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29778b20-8ecd-4310-8baa-9b7f7ad72efa None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:35:58.677945 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 639/1287] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jul 6 13:35:58 2022] GET /identity/v3/role_assignments?user.id=890d2ae150d647a18dae25159cf97484&scope.project.id=bfe6c3d8da2743be9609184db2506303&role.id=795daa2c78354fd48e4a741e1182cc98 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:36:00.738953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b90f5-2d2a-4241-895d-23a14fda138a None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:36:00.739454 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b90f5-2d2a-4241-895d-23a14fda138a None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:36:00.739454 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b90f5-2d2a-4241-895d-23a14fda138a None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:36:00.740784 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 649/1288] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:36:00.746240 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867d5e1-a140-4fdf-ab32-305fcd4e7275 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:36:00.746664 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867d5e1-a140-4fdf-ab32-305fcd4e7275 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:36:00.746796 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867d5e1-a140-4fdf-ab32-305fcd4e7275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:36:00.763135 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-f867d5e1-a140-4fdf-ab32-305fcd4e7275 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:36:00.764300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-f867d5e1-a140-4fdf-ab32-305fcd4e7275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:36:00.793766 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:36:00.830514 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 640/1289] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:36:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 13:36:00.853662 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b90f5-2d2a-4241-895d-23a14fda138a None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:36:00.952948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b90f5-2d2a-4241-895d-23a14fda138a None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:36:00.966408 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:36:00.997850 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1f04f-2efb-499e-8bb2-e428746790c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:36:00.998930 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1f04f-2efb-499e-8bb2-e428746790c2 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:36:00.999354 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1f04f-2efb-499e-8bb2-e428746790c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:36:00.999683 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1f04f-2efb-499e-8bb2-e428746790c2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:36:01.000581 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1f04f-2efb-499e-8bb2-e428746790c2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:36:01.002506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1f04f-2efb-499e-8bb2-e428746790c2 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:36:01.016505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-f1b1f04f-2efb-499e-8bb2-e428746790c2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 06 13:36:01.016505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:36:01.016505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 13:36:01.016505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 13:36:01.016505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 13:36:01.016505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 13:36:01.016505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 13:36:01.016505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:36:01.016505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 06 13:36:01.016505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:36:01.016505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 13:36:01.016505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:36:01.016505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:36:01.016505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:36:01.016505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:36:01.016505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:36:01.016505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:36:01.017708 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:36:01.017708 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:36:01.017708 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:36:01.017708 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:36:01.017708 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:36:01.017708 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:36:01.017708 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 13:36:01.017708 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 13:36:01.017708 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 13:36:01.017708 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 13:36:01.017708 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:36:01.017708 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:36:01.017708 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:36:01.017708 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:36:01.017708 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:36:01.017708 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:36:01.018682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:36:01.018682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:36:01.018682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:36:01.018682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:36:01.018682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:36:01.018682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:36:01.018682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:36:01.018682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:36:01.018682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:36:01.018682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:36:01.018682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 13:36:01.018682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:36:01.018682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 06 13:36:01.018682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:36:01.018682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 650/1290] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Jul 6 13:36:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:36:01.025506 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867d5e1-a140-4fdf-ab32-305fcd4e7275 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:36:01.133363 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867d5e1-a140-4fdf-ab32-305fcd4e7275 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:36:01.166755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2295b3-8e23-4786-a79d-caa108978665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:36:01.167414 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2295b3-8e23-4786-a79d-caa108978665 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:36:01.167698 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2295b3-8e23-4786-a79d-caa108978665 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:36:01.167976 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2295b3-8e23-4786-a79d-caa108978665 None admin] PATH_INFO: `/v3/services` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:36:01.168680 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2295b3-8e23-4786-a79d-caa108978665 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:36:01.168988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2295b3-8e23-4786-a79d-caa108978665 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:36:01.170375 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2295b3-8e23-4786-a79d-caa108978665 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:36:01.175350 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 641/1291] 10.222.0.20 () {56 vars in 1116 bytes} [Wed Jul 6 13:36:01 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:36:01.181440 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1f04f-2efb-499e-8bb2-e428746790c2 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:36:01.281997 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1f04f-2efb-499e-8bb2-e428746790c2 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:36:01.315330 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b6566a-e287-49d5-ba3e-257caa7a9954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:36:01.315993 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6566a-e287-49d5-ba3e-257caa7a9954 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:36:01.316280 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6566a-e287-49d5-ba3e-257caa7a9954 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:36:01.316579 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6566a-e287-49d5-ba3e-257caa7a9954 None admin] PATH_INFO: `/v3/services` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:36:01.317360 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b6566a-e287-49d5-ba3e-257caa7a9954 None admin] RBAC: Adding query filter params (type=placement) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:36:01.317668 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b6566a-e287-49d5-ba3e-257caa7a9954 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:36:01.319227 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b6566a-e287-49d5-ba3e-257caa7a9954 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:36:01.324631 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 651/1292] 10.222.0.20 () {56 vars in 1116 bytes} [Wed Jul 6 13:36:01 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:36:03.395466 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab40b4d-8aca-480a-b4c8-cb75d08c9c4c None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:36:03.396010 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab40b4d-8aca-480a-b4c8-cb75d08c9c4c None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:36:03.396010 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab40b4d-8aca-480a-b4c8-cb75d08c9c4c None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:36:03.398010 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 642/1293] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:36:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:36:03.403483 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26fbb3-0bc0-416c-8623-15ee882bf7a8 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:36:03.403918 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26fbb3-0bc0-416c-8623-15ee882bf7a8 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:36:03.404026 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26fbb3-0bc0-416c-8623-15ee882bf7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:36:03.420584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-3c26fbb3-0bc0-416c-8623-15ee882bf7a8 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:36:03.421789 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-3c26fbb3-0bc0-416c-8623-15ee882bf7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:36:03.450498 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:36:03.489183 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 652/1294] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:36:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:36:03.512247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab40b4d-8aca-480a-b4c8-cb75d08c9c4c None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:36:03.542906 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:36:03.612694 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab40b4d-8aca-480a-b4c8-cb75d08c9c4c None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:36:03.645945 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de6137-90ac-48cc-996e-5d42bfc42d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:36:03.646598 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de6137-90ac-48cc-996e-5d42bfc42d71 None admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:36:03.646881 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de6137-90ac-48cc-996e-5d42bfc42d71 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:36:03.647158 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de6137-90ac-48cc-996e-5d42bfc42d71 None admin] PATH_INFO: `/v3/services` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:36:03.648458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68de6137-90ac-48cc-996e-5d42bfc42d71 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:36:03.649843 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68de6137-90ac-48cc-996e-5d42bfc42d71 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:36:03.662155 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 643/1295] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Jul 6 13:36:03 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:36:05.685933 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d92b1b-756c-4476-b3b5-84ca75624b9d None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:36:05.686435 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d92b1b-756c-4476-b3b5-84ca75624b9d None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:36:05.686435 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d92b1b-756c-4476-b3b5-84ca75624b9d None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:36:05.687785 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 653/1296] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:36:05.692724 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81a1c2-458a-40b5-bcea-3d6f28091adf None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:36:05.693183 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81a1c2-458a-40b5-bcea-3d6f28091adf None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:36:05.693313 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81a1c2-458a-40b5-bcea-3d6f28091adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:36:05.711548 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-2f81a1c2-458a-40b5-bcea-3d6f28091adf None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:36:05.712731 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-2f81a1c2-458a-40b5-bcea-3d6f28091adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:36:05.782669 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 644/1297] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:36:05 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:36:05.804989 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d92b1b-756c-4476-b3b5-84ca75624b9d None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:36:05.901629 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d92b1b-756c-4476-b3b5-84ca75624b9d None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:36:05.933690 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5864e-046b-4bf9-817e-df8c0afc82b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:36:05.934464 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5864e-046b-4bf9-817e-df8c0afc82b5 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:36:05.934745 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5864e-046b-4bf9-817e-df8c0afc82b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:36:05.935018 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5864e-046b-4bf9-817e-df8c0afc82b5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:36:05.935661 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e5864e-046b-4bf9-817e-df8c0afc82b5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:36:05.937077 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e5864e-046b-4bf9-817e-df8c0afc82b5 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:36:05.949248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-f1e5864e-046b-4bf9-817e-df8c0afc82b5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 06 13:36:05.949248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:36:05.949248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 13:36:05.949248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 13:36:05.949248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 13:36:05.949248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 13:36:05.949248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 13:36:05.949248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:36:05.949248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 06 13:36:05.949248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:36:05.949248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 13:36:05.949248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:36:05.949248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:36:05.949248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:36:05.949248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:36:05.949248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:36:05.949248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:36:05.950439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:36:05.950439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:36:05.950439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:36:05.950439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:36:05.950439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:36:05.950439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:36:05.950439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 13:36:05.950439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 13:36:05.950439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 13:36:05.950439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 13:36:05.950439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:36:05.950439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:36:05.950439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:36:05.950439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:36:05.950439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:36:05.950439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:36:05.951493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:36:05.951493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:36:05.951493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:36:05.951493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:36:05.951493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:36:05.951493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:36:05.951493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:36:05.951493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:36:05.951493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:36:05.951493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:36:05.951493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 13:36:05.951493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:36:05.951493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 06 13:36:05.951493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:36:05.951493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 654/1298] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Jul 6 13:36:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:36:05.956834 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81a1c2-458a-40b5-bcea-3d6f28091adf None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:36:05.979308 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:36:06.063490 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81a1c2-458a-40b5-bcea-3d6f28091adf None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:36:06.095657 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e962031f-c226-4ca8-8fe5-dbf3f97e8415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:36:06.096309 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962031f-c226-4ca8-8fe5-dbf3f97e8415 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:36:06.096589 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962031f-c226-4ca8-8fe5-dbf3f97e8415 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:36:06.096861 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962031f-c226-4ca8-8fe5-dbf3f97e8415 None admin] PATH_INFO: `/v3/services` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:36:06.097581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e962031f-c226-4ca8-8fe5-dbf3f97e8415 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:36:06.097883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e962031f-c226-4ca8-8fe5-dbf3f97e8415 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:36:06.099240 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e962031f-c226-4ca8-8fe5-dbf3f97e8415 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:36:06.104269 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 645/1299] 10.222.0.20 () {56 vars in 1116 bytes} [Wed Jul 6 13:36:05 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:36:06.110397 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5864e-046b-4bf9-817e-df8c0afc82b5 None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:36:06.142397 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:36:06.209385 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5864e-046b-4bf9-817e-df8c0afc82b5 None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:36:06.243675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f76280-9915-4bbd-991b-0bb14d3e76f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:36:06.244328 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f76280-9915-4bbd-991b-0bb14d3e76f6 None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:36:06.244616 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f76280-9915-4bbd-991b-0bb14d3e76f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:36:06.244898 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f76280-9915-4bbd-991b-0bb14d3e76f6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:36:06.245733 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f76280-9915-4bbd-991b-0bb14d3e76f6 None admin] RBAC: Adding query filter params (interface=public, service_id=9d84e714b223400d98289bb974501842, region_id=RegionOne) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:36:06.246050 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f76280-9915-4bbd-991b-0bb14d3e76f6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:36:06.247409 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f76280-9915-4bbd-991b-0bb14d3e76f6 None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:36:06.252342 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 655/1300] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Jul 6 13:36:06 2022] GET /identity/v3/endpoints?service_id=9d84e714b223400d98289bb974501842&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:36:06.258571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e962031f-c226-4ca8-8fe5-dbf3f97e8415 None admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:36:06.337093 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:36:06.354930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e962031f-c226-4ca8-8fe5-dbf3f97e8415 None admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:36:06.387146 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e7c482-043d-4211-9cde-b9bdd92b09e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:36:06.387792 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7c482-043d-4211-9cde-b9bdd92b09e8 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:36:06.388073 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7c482-043d-4211-9cde-b9bdd92b09e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:36:06.388343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7c482-043d-4211-9cde-b9bdd92b09e8 None admin] PATH_INFO: `/v3/services` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:36:06.389023 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e7c482-043d-4211-9cde-b9bdd92b09e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:36:06.390707 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e7c482-043d-4211-9cde-b9bdd92b09e8 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:36:06.397317 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 646/1301] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Jul 6 13:36:06 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:36:08.412463 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c945818-d434-4c00-8ef6-9bcc45a9dc51 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:36:08.413068 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c945818-d434-4c00-8ef6-9bcc45a9dc51 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:36:08.413068 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c945818-d434-4c00-8ef6-9bcc45a9dc51 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:36:08.414959 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 656/1302] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:36:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:36:08.419988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7a95b-2185-4cda-82ef-a66e1305fd76 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:36:08.420395 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7a95b-2185-4cda-82ef-a66e1305fd76 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:36:08.420553 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7a95b-2185-4cda-82ef-a66e1305fd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:36:08.437292 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-fdf7a95b-2185-4cda-82ef-a66e1305fd76 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:36:08.438465 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-fdf7a95b-2185-4cda-82ef-a66e1305fd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:36:08.503061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 647/1303] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:36:08 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 13:36:08.526087 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c945818-d434-4c00-8ef6-9bcc45a9dc51 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:36:08.617454 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:36:08.624190 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c945818-d434-4c00-8ef6-9bcc45a9dc51 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:36:08.656552 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d14e3c-3a91-4e3f-a0b5-6d7a20df0fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:36:08.657358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d14e3c-3a91-4e3f-a0b5-6d7a20df0fcb None admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:36:08.657642 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d14e3c-3a91-4e3f-a0b5-6d7a20df0fcb None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:36:08.657919 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d14e3c-3a91-4e3f-a0b5-6d7a20df0fcb None admin] PATH_INFO: `/v3/services/placement` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:36:08.658567 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d14e3c-3a91-4e3f-a0b5-6d7a20df0fcb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:36:08.659914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d14e3c-3a91-4e3f-a0b5-6d7a20df0fcb None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:36:08.671513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application [None req-c2d14e3c-3a91-4e3f-a0b5-6d7a20df0fcb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 06 13:36:08.671513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:36:08.671513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 13:36:08.671513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 13:36:08.671513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 13:36:08.671513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 13:36:08.671513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 13:36:08.671513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:36:08.671513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 06 13:36:08.671513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:36:08.671513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 13:36:08.671513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:36:08.671513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 13:36:08.671513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 13:36:08.671513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 13:36:08.671513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 13:36:08.671513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 13:36:08.672911 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 13:36:08.672911 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 13:36:08.672911 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 13:36:08.672911 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 13:36:08.672911 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 13:36:08.672911 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 13:36:08.672911 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 13:36:08.672911 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 13:36:08.672911 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 13:36:08.672911 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 13:36:08.672911 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 13:36:08.672911 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 13:36:08.672911 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 13:36:08.672911 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 13:36:08.672911 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 13:36:08.672911 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 13:36:08.674162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 13:36:08.674162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application with Lock( Jul 06 13:36:08.674162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 13:36:08.674162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self._enter() Jul 06 13:36:08.674162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 13:36:08.674162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 13:36:08.674162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 13:36:08.674162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application return self.creator() Jul 06 13:36:08.674162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 13:36:08.674162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application created_value = creator( Jul 06 13:36:08.674162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 13:36:08.674162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 13:36:08.674162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 06 13:36:08.674162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: ERROR keystone.server.flask.application Jul 06 13:36:08.674162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 657/1304] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Jul 6 13:36:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 13:36:08.678885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7a95b-2185-4cda-82ef-a66e1305fd76 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:36:08.752807 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:36:08.776010 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7a95b-2185-4cda-82ef-a66e1305fd76 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:36:08.808505 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d8ff9-d5af-465e-84f7-b7bd475faa14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:36:08.809184 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d8ff9-d5af-465e-84f7-b7bd475faa14 None admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:36:08.809469 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d8ff9-d5af-465e-84f7-b7bd475faa14 None admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:36:08.809740 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d8ff9-d5af-465e-84f7-b7bd475faa14 None admin] PATH_INFO: `/v3/services` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:36:08.810433 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d8ff9-d5af-465e-84f7-b7bd475faa14 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:36:08.810736 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d8ff9-d5af-465e-84f7-b7bd475faa14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:36:08.812124 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d8ff9-d5af-465e-84f7-b7bd475faa14 None admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:36:08.817071 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 648/1305] 10.222.0.20 () {56 vars in 1116 bytes} [Wed Jul 6 13:36:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:36:08.823275 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d14e3c-3a91-4e3f-a0b5-6d7a20df0fcb None admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:36:08.920358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d14e3c-3a91-4e3f-a0b5-6d7a20df0fcb None admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:36:08.953686 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442fb735-b947-46cc-a2bf-7563a5291fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:36:08.954321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fb735-b947-46cc-a2bf-7563a5291fcc None admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:36:08.954608 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fb735-b947-46cc-a2bf-7563a5291fcc None admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:36:08.954887 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fb735-b947-46cc-a2bf-7563a5291fcc None admin] PATH_INFO: `/v3/endpoints` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:36:08.955622 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442fb735-b947-46cc-a2bf-7563a5291fcc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:36:08.956974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442fb735-b947-46cc-a2bf-7563a5291fcc None admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:36:08.974267 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 658/1306] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Jul 6 13:36:08 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:36:18.814187 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=30 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:36:18.970058 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:37:12.335175 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059dcd3-8699-4404-b985-765f518d3e2b None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:12.335734 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059dcd3-8699-4404-b985-765f518d3e2b None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:12.335734 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059dcd3-8699-4404-b985-765f518d3e2b None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:12.340733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 649/1307] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:37:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:37:12.342269 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc7aa8-5d8b-4402-b84a-3fb929e1a25d None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:12.342689 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc7aa8-5d8b-4402-b84a-3fb929e1a25d None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:12.342869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc7aa8-5d8b-4402-b84a-3fb929e1a25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:12.433677 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-6bfc7aa8-5d8b-4402-b84a-3fb929e1a25d None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:37:12.434869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-6bfc7aa8-5d8b-4402-b84a-3fb929e1a25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:12.549925 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 659/1308] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:37:12.600600 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041a5bc-4e92-4582-a38d-2e7696ec197c None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:12.600960 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041a5bc-4e92-4582-a38d-2e7696ec197c None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:12.601455 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041a5bc-4e92-4582-a38d-2e7696ec197c None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:12.602818 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 650/1309] 10.222.0.20 () {54 vars in 964 bytes} [Wed Jul 6 13:37:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:37:12.618193 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4525cb2-9f46-4663-afa0-7fe3700e0daf None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:12.618920 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4525cb2-9f46-4663-afa0-7fe3700e0daf None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:12.619845 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4525cb2-9f46-4663-afa0-7fe3700e0daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:12.788749 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-b4525cb2-9f46-4663-afa0-7fe3700e0daf None None] MFA Rules not processed for user `b9f250f5dbc54d2e9293962ece848f6c`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:37:12.790173 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-b4525cb2-9f46-4663-afa0-7fe3700e0daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:12.832020 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:37:12.898705 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 660/1310] 10.222.0.20 () {58 vars in 1061 bytes} [Wed Jul 6 13:37:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:37:12.908085 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041a5bc-4e92-4582-a38d-2e7696ec197c None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:37:12.911575 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a041a5bc-4e92-4582-a38d-2e7696ec197c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:13.024453 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041a5bc-4e92-4582-a38d-2e7696ec197c None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:37:13.101856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd9ae1f-c382-41b3-b666-edd7d0352a04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:37:13.102423 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9ae1f-c382-41b3-b666-edd7d0352a04 service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:13.102710 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9ae1f-c382-41b3-b666-edd7d0352a04 service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:13.103001 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9ae1f-c382-41b3-b666-edd7d0352a04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:13.104658 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-bbd9ae1f-c382-41b3-b666-edd7d0352a04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:13.163939 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:37:13.223574 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd9ae1f-c382-41b3-b666-edd7d0352a04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:37:13.225048 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd9ae1f-c382-41b3-b666-edd7d0352a04 service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:37:13.244435 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 651/1311] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:37:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:37:15.571722 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c6b41-73e5-4c82-9be0-599008928f40 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:15.572226 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c6b41-73e5-4c82-9be0-599008928f40 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:15.572226 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c6b41-73e5-4c82-9be0-599008928f40 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:15.573599 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 661/1312] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:37:15.578885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4f8c8-abdc-4779-bce2-4b84a9853d9d None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:15.579285 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4f8c8-abdc-4779-bce2-4b84a9853d9d None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:15.579460 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4f8c8-abdc-4779-bce2-4b84a9853d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:15.648496 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-adb4f8c8-abdc-4779-bce2-4b84a9853d9d None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:37:15.649710 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-adb4f8c8-abdc-4779-bce2-4b84a9853d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:15.747597 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 652/1313] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:37:15.786569 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c6b41-73e5-4c82-9be0-599008928f40 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:37:15.788127 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-eb4c6b41-73e5-4c82-9be0-599008928f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:15.897910 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c6b41-73e5-4c82-9be0-599008928f40 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:37:15.983884 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:37:15.986690 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2512a-29f0-45ea-9092-cc7041e09446 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:37:15.988214 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2512a-29f0-45ea-9092-cc7041e09446 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:15.988214 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2512a-29f0-45ea-9092-cc7041e09446 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:15.988214 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2512a-29f0-45ea-9092-cc7041e09446 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:15.990650 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-a8b2512a-29f0-45ea-9092-cc7041e09446 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:16.107551 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b2512a-29f0-45ea-9092-cc7041e09446 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:37:16.109030 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b2512a-29f0-45ea-9092-cc7041e09446 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:37:16.128051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 662/1314] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:37:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:37:18.500714 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3203e5d-0ddc-4511-b2c9-51714b7b3b8c None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:18.501298 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3203e5d-0ddc-4511-b2c9-51714b7b3b8c None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:18.501298 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3203e5d-0ddc-4511-b2c9-51714b7b3b8c None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:18.502695 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 653/1315] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:37:18.507870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b0c18-34cc-452e-a4fa-c7af2fc0ba8c None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:18.508269 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b0c18-34cc-452e-a4fa-c7af2fc0ba8c None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:18.508430 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b0c18-34cc-452e-a4fa-c7af2fc0ba8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:18.572055 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-da6b0c18-34cc-452e-a4fa-c7af2fc0ba8c None None] MFA Rules not processed for user `31e4ee38bf8148f4a4dd9f56097514a1`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:37:18.573185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-da6b0c18-34cc-452e-a4fa-c7af2fc0ba8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:18.622989 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:37:18.666311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 663/1316] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:37:18 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:37:18.707151 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3203e5d-0ddc-4511-b2c9-51714b7b3b8c None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:37:18.708676 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a3203e5d-0ddc-4511-b2c9-51714b7b3b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:18.718287 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:37:18.809267 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3203e5d-0ddc-4511-b2c9-51714b7b3b8c None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:37:18.881598 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4550c1c9-5d4f-449e-a912-7858deab61bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:37:18.882165 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550c1c9-5d4f-449e-a912-7858deab61bd service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:18.882444 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550c1c9-5d4f-449e-a912-7858deab61bd service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:18.882721 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550c1c9-5d4f-449e-a912-7858deab61bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:18.884344 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-4550c1c9-5d4f-449e-a912-7858deab61bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:18.944568 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:37:18.995688 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4550c1c9-5d4f-449e-a912-7858deab61bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:37:18.997162 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4550c1c9-5d4f-449e-a912-7858deab61bd service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:37:19.015829 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 654/1317] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:37:18 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:37:21.605645 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458db529-23ff-4860-bf1b-edc01b4678d6 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:21.606147 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458db529-23ff-4860-bf1b-edc01b4678d6 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:21.606147 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458db529-23ff-4860-bf1b-edc01b4678d6 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:21.607480 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 664/1318] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:37:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:37:21.613240 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36309531-0afd-4147-bbc0-ed9a236b04c1 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:21.613776 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36309531-0afd-4147-bbc0-ed9a236b04c1 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:21.613776 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36309531-0afd-4147-bbc0-ed9a236b04c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:21.684413 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-36309531-0afd-4147-bbc0-ed9a236b04c1 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:37:21.685204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-36309531-0afd-4147-bbc0-ed9a236b04c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:21.781797 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 655/1319] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:37:21 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:37:21.820218 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458db529-23ff-4860-bf1b-edc01b4678d6 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:37:21.821793 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-458db529-23ff-4860-bf1b-edc01b4678d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:21.925140 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458db529-23ff-4860-bf1b-edc01b4678d6 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:37:22.019584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83410b87-b037-424f-962e-e7a73e884364 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:37:22.020254 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83410b87-b037-424f-962e-e7a73e884364 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:22.020539 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83410b87-b037-424f-962e-e7a73e884364 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:22.020820 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83410b87-b037-424f-962e-e7a73e884364 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:22.022499 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-83410b87-b037-424f-962e-e7a73e884364 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:22.030792 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:37:22.130402 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83410b87-b037-424f-962e-e7a73e884364 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:37:22.131828 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83410b87-b037-424f-962e-e7a73e884364 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:37:22.149874 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 665/1320] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:37:21 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:37:24.838376 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9407e-607a-4327-ad34-96be3b247094 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:24.838917 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9407e-607a-4327-ad34-96be3b247094 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:24.838917 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9407e-607a-4327-ad34-96be3b247094 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:24.840294 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 656/1321] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:37:24.845655 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b11502-d0dc-4469-8fc0-9e9c80adcc1a None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:24.846075 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b11502-d0dc-4469-8fc0-9e9c80adcc1a None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:24.846218 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b11502-d0dc-4469-8fc0-9e9c80adcc1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:24.910543 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-b0b11502-d0dc-4469-8fc0-9e9c80adcc1a None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:37:24.911766 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-b0b11502-d0dc-4469-8fc0-9e9c80adcc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:24.968718 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:37:25.010661 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 666/1322] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:37:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:37:25.048179 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f9407e-607a-4327-ad34-96be3b247094 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:37:25.050043 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-47f9407e-607a-4327-ad34-96be3b247094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:25.072717 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:37:25.161815 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f9407e-607a-4327-ad34-96be3b247094 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:37:25.239695 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4d084-d144-424f-8eb2-3733227779bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:37:25.240356 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4d084-d144-424f-8eb2-3733227779bc service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:25.240644 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4d084-d144-424f-8eb2-3733227779bc service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:25.240922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4d084-d144-424f-8eb2-3733227779bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:25.242596 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-02d4d084-d144-424f-8eb2-3733227779bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:25.317498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:37:25.356282 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d4d084-d144-424f-8eb2-3733227779bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:37:25.357754 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d4d084-d144-424f-8eb2-3733227779bc service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:37:25.376205 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 657/1323] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:37:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:37:27.534367 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c89c6-739e-4a01-8024-902100471c35 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:27.534902 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c89c6-739e-4a01-8024-902100471c35 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:27.535014 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c89c6-739e-4a01-8024-902100471c35 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:27.536410 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 667/1324] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:37:27.541702 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84d42f-5400-463e-8a8e-22d40977f0de None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:27.542124 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84d42f-5400-463e-8a8e-22d40977f0de None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:27.542267 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84d42f-5400-463e-8a8e-22d40977f0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:27.608786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-cd84d42f-5400-463e-8a8e-22d40977f0de None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:37:27.610012 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-cd84d42f-5400-463e-8a8e-22d40977f0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:27.714649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 658/1325] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:37:27.749901 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c89c6-739e-4a01-8024-902100471c35 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:37:27.751437 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-a76c89c6-739e-4a01-8024-902100471c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:27.858867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c89c6-739e-4a01-8024-902100471c35 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:37:27.936084 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b9a5a9-bdee-44ce-b54e-c2ecb5ee527c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:37:27.936746 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9a5a9-bdee-44ce-b54e-c2ecb5ee527c service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:27.937059 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9a5a9-bdee-44ce-b54e-c2ecb5ee527c service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:27.937343 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9a5a9-bdee-44ce-b54e-c2ecb5ee527c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:27.939025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-a1b9a5a9-bdee-44ce-b54e-c2ecb5ee527c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:27.957859 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:37:28.052498 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b9a5a9-bdee-44ce-b54e-c2ecb5ee527c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:37:28.053972 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b9a5a9-bdee-44ce-b54e-c2ecb5ee527c service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:37:28.072353 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 668/1326] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:37:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:37:30.183501 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290ed83-5d54-4568-8ff1-5d5f3cdecdf7 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:30.184091 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290ed83-5d54-4568-8ff1-5d5f3cdecdf7 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:30.184091 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290ed83-5d54-4568-8ff1-5d5f3cdecdf7 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:30.185468 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 659/1327] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:37:30.190873 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185bd857-c7d6-4ffd-90d8-e381e90b5d55 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:30.191261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185bd857-c7d6-4ffd-90d8-e381e90b5d55 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:30.191374 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185bd857-c7d6-4ffd-90d8-e381e90b5d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:30.259055 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-185bd857-c7d6-4ffd-90d8-e381e90b5d55 None None] MFA Rules not processed for user `31e4ee38bf8148f4a4dd9f56097514a1`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:37:30.260278 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-185bd857-c7d6-4ffd-90d8-e381e90b5d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:30.328238 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:37:30.362438 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 669/1328] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:37:30 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:37:30.399848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290ed83-5d54-4568-8ff1-5d5f3cdecdf7 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:37:30.401424 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-1290ed83-5d54-4568-8ff1-5d5f3cdecdf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:30.431619 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:37:30.506510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290ed83-5d54-4568-8ff1-5d5f3cdecdf7 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:37:30.582766 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c534084-cd7b-4cd6-832c-b898451cc337 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:37:30.583421 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c534084-cd7b-4cd6-832c-b898451cc337 service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:30.583701 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c534084-cd7b-4cd6-832c-b898451cc337 service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:30.583973 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c534084-cd7b-4cd6-832c-b898451cc337 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:30.585641 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-8c534084-cd7b-4cd6-832c-b898451cc337 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:30.660716 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:37:30.693765 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c534084-cd7b-4cd6-832c-b898451cc337 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:37:30.695207 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c534084-cd7b-4cd6-832c-b898451cc337 service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:37:30.727713 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 660/1329] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:37:30 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:37:33.065802 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f59779e-cca6-45b0-b579-2c57c9a843a7 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:33.066445 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f59779e-cca6-45b0-b579-2c57c9a843a7 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:33.066445 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f59779e-cca6-45b0-b579-2c57c9a843a7 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:33.067826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 670/1330] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:37:33.073031 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be7514-e34d-4cc4-a23a-8c954ba74913 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:33.073466 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be7514-e34d-4cc4-a23a-8c954ba74913 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:33.073587 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be7514-e34d-4cc4-a23a-8c954ba74913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:33.137363 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-a6be7514-e34d-4cc4-a23a-8c954ba74913 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:37:33.138536 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a6be7514-e34d-4cc4-a23a-8c954ba74913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:33.233703 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 661/1331] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:37:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:37:33.270008 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f59779e-cca6-45b0-b579-2c57c9a843a7 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:37:33.271515 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-8f59779e-cca6-45b0-b579-2c57c9a843a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:33.373463 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f59779e-cca6-45b0-b579-2c57c9a843a7 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:37:33.444254 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9520379-84a7-4e14-9530-324ceeea809a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:37:33.444891 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9520379-84a7-4e14-9530-324ceeea809a service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:33.445201 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9520379-84a7-4e14-9530-324ceeea809a service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:33.445471 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9520379-84a7-4e14-9530-324ceeea809a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:33.447093 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-f9520379-84a7-4e14-9530-324ceeea809a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:33.472960 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:37:33.552291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9520379-84a7-4e14-9530-324ceeea809a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:37:33.553737 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9520379-84a7-4e14-9530-324ceeea809a service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:37:33.571514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 671/1332] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:37:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:37:35.686500 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941e787-ad7b-4c5c-8cdd-98cf22519446 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:35.687084 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941e787-ad7b-4c5c-8cdd-98cf22519446 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:35.687084 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941e787-ad7b-4c5c-8cdd-98cf22519446 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:35.688345 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 662/1333] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:37:35.693789 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc0e45-41f0-4c70-a402-dd359ba60251 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:35.694243 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc0e45-41f0-4c70-a402-dd359ba60251 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:35.694355 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc0e45-41f0-4c70-a402-dd359ba60251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:35.764551 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-bacc0e45-41f0-4c70-a402-dd359ba60251 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:37:35.765787 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-bacc0e45-41f0-4c70-a402-dd359ba60251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:35.837900 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:37:35.860000 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 672/1334] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:37:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:37:35.899771 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c941e787-ad7b-4c5c-8cdd-98cf22519446 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:37:35.901339 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-c941e787-ad7b-4c5c-8cdd-98cf22519446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:35.941768 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:37:36.004729 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c941e787-ad7b-4c5c-8cdd-98cf22519446 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:37:36.077547 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ef967-afc8-47fa-90d1-4ac66f941e58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:37:36.078197 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ef967-afc8-47fa-90d1-4ac66f941e58 service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:36.078483 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ef967-afc8-47fa-90d1-4ac66f941e58 service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:36.078760 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ef967-afc8-47fa-90d1-4ac66f941e58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:36.080390 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-177ef967-afc8-47fa-90d1-4ac66f941e58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:36.174314 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:37:36.200207 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ef967-afc8-47fa-90d1-4ac66f941e58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:37:36.201685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ef967-afc8-47fa-90d1-4ac66f941e58 service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:37:36.219719 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 663/1335] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:37:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:37:38.692474 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b65c2-b8ff-4017-8759-a730e6760d40 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:38.693059 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b65c2-b8ff-4017-8759-a730e6760d40 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:38.693059 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b65c2-b8ff-4017-8759-a730e6760d40 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:38.694419 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 673/1336] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:37:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:37:38.699859 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a239d-8ed8-4238-83e9-f6f24d908d40 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:38.700296 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a239d-8ed8-4238-83e9-f6f24d908d40 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:38.700401 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a239d-8ed8-4238-83e9-f6f24d908d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:38.764187 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-768a239d-8ed8-4238-83e9-f6f24d908d40 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:37:38.765409 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-768a239d-8ed8-4238-83e9-f6f24d908d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:38.865431 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 664/1337] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:37:38 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:37:38.913951 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3785437-ade7-494a-ae6c-a126d8b36f76 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:38.914237 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3785437-ade7-494a-ae6c-a126d8b36f76 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:38.914514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3785437-ade7-494a-ae6c-a126d8b36f76 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:38.915905 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 674/1338] 10.222.0.20 () {54 vars in 964 bytes} [Wed Jul 6 13:37:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:37:38.921471 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab0a0c-6c58-4ca2-bb3d-f857e1e44aca None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:38.922020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab0a0c-6c58-4ca2-bb3d-f857e1e44aca None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:38.922302 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab0a0c-6c58-4ca2-bb3d-f857e1e44aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:38.978691 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:37:39.001117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-28ab0a0c-6c58-4ca2-bb3d-f857e1e44aca None None] MFA Rules not processed for user `b9f250f5dbc54d2e9293962ece848f6c`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:37:39.002305 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-28ab0a0c-6c58-4ca2-bb3d-f857e1e44aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:39.076113 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 665/1339] 10.222.0.20 () {58 vars in 1061 bytes} [Wed Jul 6 13:37:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:37:39.086938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3785437-ade7-494a-ae6c-a126d8b36f76 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:37:39.088576 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-a3785437-ade7-494a-ae6c-a126d8b36f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:39.190657 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3785437-ade7-494a-ae6c-a126d8b36f76 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:37:39.261376 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85fd628-cc60-403c-b2ac-0e1d0c9e0c58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:37:39.262021 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fd628-cc60-403c-b2ac-0e1d0c9e0c58 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:39.262304 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fd628-cc60-403c-b2ac-0e1d0c9e0c58 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:39.262576 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fd628-cc60-403c-b2ac-0e1d0c9e0c58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:39.264207 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-c85fd628-cc60-403c-b2ac-0e1d0c9e0c58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:39.299566 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:37:39.372103 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85fd628-cc60-403c-b2ac-0e1d0c9e0c58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:37:39.373557 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85fd628-cc60-403c-b2ac-0e1d0c9e0c58 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:37:39.391782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 675/1340] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:37:39 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:37:43.912605 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ff626-41fc-4140-9829-b9cb5db85ed8 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:43.913922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ff626-41fc-4140-9829-b9cb5db85ed8 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:43.914382 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ff626-41fc-4140-9829-b9cb5db85ed8 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:43.916302 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 666/1341] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:37:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:37:43.921572 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4e4d4-7af6-44d6-a8d1-784449a7fe2d None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:43.922502 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4e4d4-7af6-44d6-a8d1-784449a7fe2d None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:43.922794 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4e4d4-7af6-44d6-a8d1-784449a7fe2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:44.007532 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-31e4e4d4-7af6-44d6-a8d1-784449a7fe2d None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:37:44.008797 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-31e4e4d4-7af6-44d6-a8d1-784449a7fe2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:44.103712 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:37:44.117723 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 676/1342] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:37:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:37:44.157241 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ff626-41fc-4140-9829-b9cb5db85ed8 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:37:44.159797 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-6d7ff626-41fc-4140-9829-b9cb5db85ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:44.265694 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ff626-41fc-4140-9829-b9cb5db85ed8 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:37:44.269993 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce82a28-6fa0-4cbe-a84c-4833bffa7fea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:37:44.270644 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce82a28-6fa0-4cbe-a84c-4833bffa7fea service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:44.270927 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce82a28-6fa0-4cbe-a84c-4833bffa7fea service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:44.271198 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce82a28-6fa0-4cbe-a84c-4833bffa7fea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:44.280650 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce82a28-6fa0-4cbe-a84c-4833bffa7fea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:37:44.282118 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce82a28-6fa0-4cbe-a84c-4833bffa7fea service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:37:44.292976 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 667/1343] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:37:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:37:46.972670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86065cff-424b-44ab-b3af-65dadd9d751a None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:46.973328 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86065cff-424b-44ab-b3af-65dadd9d751a None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:46.973328 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86065cff-424b-44ab-b3af-65dadd9d751a None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:46.975154 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 677/1344] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:37:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:37:46.981358 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeec5df-7502-4ec8-92a9-bee9b05df8c0 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:46.981794 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeec5df-7502-4ec8-92a9-bee9b05df8c0 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:46.982046 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeec5df-7502-4ec8-92a9-bee9b05df8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:47.010464 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-cbeec5df-7502-4ec8-92a9-bee9b05df8c0 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:37:47.011661 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-cbeec5df-7502-4ec8-92a9-bee9b05df8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:47.039578 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:37:47.092289 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 668/1345] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:37:46 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:37:47.131362 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86065cff-424b-44ab-b3af-65dadd9d751a None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:37:47.133164 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-86065cff-424b-44ab-b3af-65dadd9d751a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:47.146901 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86065cff-424b-44ab-b3af-65dadd9d751a None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:37:47.150394 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85b81b-736c-4d80-be5d-ba08103d5205 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:37:47.151057 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85b81b-736c-4d80-be5d-ba08103d5205 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:47.151337 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85b81b-736c-4d80-be5d-ba08103d5205 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:47.151614 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85b81b-736c-4d80-be5d-ba08103d5205 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:47.161404 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db85b81b-736c-4d80-be5d-ba08103d5205 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:37:47.162856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db85b81b-736c-4d80-be5d-ba08103d5205 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:37:47.173164 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 678/1346] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:37:47 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:37:51.157782 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01c9f1-2649-4b08-8a47-53762970b7c5 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:51.158373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01c9f1-2649-4b08-8a47-53762970b7c5 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:51.158373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01c9f1-2649-4b08-8a47-53762970b7c5 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:51.161157 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 669/1347] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:37:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:37:51.166584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47708b21-4bbb-463f-8055-2103d2caacf2 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:51.167224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47708b21-4bbb-463f-8055-2103d2caacf2 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:51.167804 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47708b21-4bbb-463f-8055-2103d2caacf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:51.187289 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-47708b21-4bbb-463f-8055-2103d2caacf2 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:37:51.188485 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-47708b21-4bbb-463f-8055-2103d2caacf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:51.196607 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 679/1348] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:37:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:37:51.234247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a01c9f1-2649-4b08-8a47-53762970b7c5 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:37:51.245319 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a01c9f1-2649-4b08-8a47-53762970b7c5 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:37:51.248897 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d87e41b-f81b-4bc9-b821-a4e23f0a1b4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:37:51.249643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d87e41b-f81b-4bc9-b821-a4e23f0a1b4f service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:51.249936 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d87e41b-f81b-4bc9-b821-a4e23f0a1b4f service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:51.250210 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d87e41b-f81b-4bc9-b821-a4e23f0a1b4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:51.259413 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d87e41b-f81b-4bc9-b821-a4e23f0a1b4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:37:51.260842 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d87e41b-f81b-4bc9-b821-a4e23f0a1b4f service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:37:51.271184 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 670/1349] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:37:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:37:56.293568 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5a78d-44bd-40f6-8cf4-c903162f185c None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:56.294388 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5a78d-44bd-40f6-8cf4-c903162f185c None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:56.294920 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5a78d-44bd-40f6-8cf4-c903162f185c None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:56.296713 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 680/1350] 10.222.0.20 () {54 vars in 921 bytes} [Wed Jul 6 13:37:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:37:56.302840 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400320ab-8d4e-4931-8fc8-12c15223c14f None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:37:56.303261 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400320ab-8d4e-4931-8fc8-12c15223c14f None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:37:56.303476 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400320ab-8d4e-4931-8fc8-12c15223c14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:37:56.431889 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-400320ab-8d4e-4931-8fc8-12c15223c14f None None] MFA Rules not processed for user `890d2ae150d647a18dae25159cf97484`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:37:56.433556 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-400320ab-8d4e-4931-8fc8-12c15223c14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:37:56.512530 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 671/1351] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Jul 6 13:37:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:38:00.721568 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef3505-4379-4e22-9438-2eb6639542b7 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:00.722194 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef3505-4379-4e22-9438-2eb6639542b7 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:00.722194 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef3505-4379-4e22-9438-2eb6639542b7 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:00.723777 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 681/1352] 10.222.0.20 () {54 vars in 921 bytes} [Wed Jul 6 13:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:38:00.729072 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9439b-7fe1-446b-a0b0-03e71e193dc9 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:00.729567 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9439b-7fe1-446b-a0b0-03e71e193dc9 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:00.729653 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9439b-7fe1-446b-a0b0-03e71e193dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:00.751093 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-04d9439b-7fe1-446b-a0b0-03e71e193dc9 None None] MFA Rules not processed for user `890d2ae150d647a18dae25159cf97484`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:38:00.753344 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-04d9439b-7fe1-446b-a0b0-03e71e193dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:38:00.763444 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 672/1353] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Jul 6 13:38:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:38:01.182120 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=13 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:38:02.376929 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffde07-154f-45ce-988a-e6e7bcbd08a7 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:02.378259 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffde07-154f-45ce-988a-e6e7bcbd08a7 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:02.378259 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffde07-154f-45ce-988a-e6e7bcbd08a7 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:02.380385 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 682/1354] 10.222.0.20 () {54 vars in 921 bytes} [Wed Jul 6 13:38:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:38:02.387560 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff068bd6-80c4-4e34-bf4a-2d08ebcb1810 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:02.388007 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff068bd6-80c4-4e34-bf4a-2d08ebcb1810 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:02.388293 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff068bd6-80c4-4e34-bf4a-2d08ebcb1810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:02.409087 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-ff068bd6-80c4-4e34-bf4a-2d08ebcb1810 None None] MFA Rules not processed for user `890d2ae150d647a18dae25159cf97484`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:38:02.410331 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-ff068bd6-80c4-4e34-bf4a-2d08ebcb1810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:38:02.419502 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 673/1355] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Jul 6 13:38:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:38:05.953285 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0ddea-b8aa-47f9-b87f-63261e6061fa None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:05.953862 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0ddea-b8aa-47f9-b87f-63261e6061fa None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:05.953862 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0ddea-b8aa-47f9-b87f-63261e6061fa None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:05.955218 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 683/1356] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:38:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:38:05.961608 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fdace-66a7-4348-9856-930e0c244dde None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:05.962101 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fdace-66a7-4348-9856-930e0c244dde None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:05.962101 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fdace-66a7-4348-9856-930e0c244dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:05.992774 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-c13fdace-66a7-4348-9856-930e0c244dde None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:38:05.997425 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-c13fdace-66a7-4348-9856-930e0c244dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:38:06.016964 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 674/1357] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:38:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:38:06.087807 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4336f-3712-4535-9303-71b246c80366 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:06.088571 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4336f-3712-4535-9303-71b246c80366 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:06.089064 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4336f-3712-4535-9303-71b246c80366 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:06.090837 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 684/1358] 10.222.0.20 () {54 vars in 961 bytes} [Wed Jul 6 13:38:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:38:06.103420 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5333e50-9c1f-4c04-9b01-36baa9d77318 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:06.104033 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5333e50-9c1f-4c04-9b01-36baa9d77318 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:06.104478 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5333e50-9c1f-4c04-9b01-36baa9d77318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:06.150695 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:38:06.161021 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-b5333e50-9c1f-4c04-9b01-36baa9d77318 None None] MFA Rules not processed for user `6647eb1e8a1544ef9d3a9806f2d1fc51`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:38:06.162209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-b5333e50-9c1f-4c04-9b01-36baa9d77318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:38:06.248844 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 675/1359] 10.222.0.20 () {58 vars in 1058 bytes} [Wed Jul 6 13:38:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:38:06.256792 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4336f-3712-4535-9303-71b246c80366 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:38:06.272007 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4336f-3712-4535-9303-71b246c80366 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:38:06.277537 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e36ca2-ef99-47ba-b1af-52a5909333ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:38:06.278282 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e36ca2-ef99-47ba-b1af-52a5909333ec service nova] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:06.278667 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e36ca2-ef99-47ba-b1af-52a5909333ec service nova] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:06.279046 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e36ca2-ef99-47ba-b1af-52a5909333ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:06.296072 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e36ca2-ef99-47ba-b1af-52a5909333ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:38:06.298098 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e36ca2-ef99-47ba-b1af-52a5909333ec service nova] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:38:06.323260 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 685/1360] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:38:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 13:38:08.631857 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0644c68-8aa8-479c-a060-c84ba5dd327a None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:08.632379 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0644c68-8aa8-479c-a060-c84ba5dd327a None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:08.632494 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0644c68-8aa8-479c-a060-c84ba5dd327a None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:08.634003 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 676/1361] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:38:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:38:08.639330 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3439f-6675-4a9a-b95d-69d9c4be7e14 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:08.639770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3439f-6675-4a9a-b95d-69d9c4be7e14 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:08.639962 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3439f-6675-4a9a-b95d-69d9c4be7e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:08.658254 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-96b3439f-6675-4a9a-b95d-69d9c4be7e14 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:38:08.659505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-96b3439f-6675-4a9a-b95d-69d9c4be7e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:38:08.667511 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 686/1362] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:38:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:38:08.713262 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0644c68-8aa8-479c-a060-c84ba5dd327a None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:38:08.728078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0644c68-8aa8-479c-a060-c84ba5dd327a None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:38:08.733967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c7f2e5-05a9-40ad-885a-c37b320633ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:38:08.734848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7f2e5-05a9-40ad-885a-c37b320633ca service nova] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:08.735323 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7f2e5-05a9-40ad-885a-c37b320633ca service nova] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:08.735804 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7f2e5-05a9-40ad-885a-c37b320633ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:08.745351 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c7f2e5-05a9-40ad-885a-c37b320633ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:38:08.747331 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c7f2e5-05a9-40ad-885a-c37b320633ca service nova] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:38:08.758855 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 677/1363] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:38:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:38:11.176592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332af25-7895-470f-b45b-77e73692a6cf None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:11.177132 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332af25-7895-470f-b45b-77e73692a6cf None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:11.177132 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332af25-7895-470f-b45b-77e73692a6cf None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:11.178795 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 687/1364] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:38:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:38:11.184219 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7742a5-7f32-4b93-9245-e74d16cd2a3e None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:11.185146 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7742a5-7f32-4b93-9245-e74d16cd2a3e None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:11.185659 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7742a5-7f32-4b93-9245-e74d16cd2a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:11.208563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-fd7742a5-7f32-4b93-9245-e74d16cd2a3e None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:38:11.210121 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-fd7742a5-7f32-4b93-9245-e74d16cd2a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:38:11.218338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 678/1365] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:38:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:38:11.263219 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46c9e8-8122-46ca-9ea9-1fe12bccdecf None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:11.263506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46c9e8-8122-46ca-9ea9-1fe12bccdecf None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:11.263787 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46c9e8-8122-46ca-9ea9-1fe12bccdecf None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:11.265207 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 688/1366] 10.222.0.20 () {54 vars in 961 bytes} [Wed Jul 6 13:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:38:11.271324 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46447f7c-e591-49c2-8cd9-f35968c565e9 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:11.271616 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46447f7c-e591-49c2-8cd9-f35968c565e9 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:11.271895 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46447f7c-e591-49c2-8cd9-f35968c565e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:11.291032 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-46447f7c-e591-49c2-8cd9-f35968c565e9 None None] MFA Rules not processed for user `6647eb1e8a1544ef9d3a9806f2d1fc51`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:38:11.292219 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-46447f7c-e591-49c2-8cd9-f35968c565e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:38:11.302412 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 679/1367] 10.222.0.20 () {58 vars in 1058 bytes} [Wed Jul 6 13:38:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:38:11.309730 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46c9e8-8122-46ca-9ea9-1fe12bccdecf None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:38:11.324064 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46c9e8-8122-46ca-9ea9-1fe12bccdecf None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:38:11.329492 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d3253f-b637-47e7-82c1-5269ba0d2a84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:38:11.330125 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3253f-b637-47e7-82c1-5269ba0d2a84 service nova] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:11.330407 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3253f-b637-47e7-82c1-5269ba0d2a84 service nova] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:11.331204 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3253f-b637-47e7-82c1-5269ba0d2a84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:11.339274 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d3253f-b637-47e7-82c1-5269ba0d2a84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:38:11.340820 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d3253f-b637-47e7-82c1-5269ba0d2a84 service nova] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:38:11.350595 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 689/1368] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:38:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:38:13.486175 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aef7cf-bde8-45ad-9576-900589bbd485 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:13.487605 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aef7cf-bde8-45ad-9576-900589bbd485 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:13.487891 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aef7cf-bde8-45ad-9576-900589bbd485 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:13.489475 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 680/1369] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:38:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:38:13.495726 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82549ef9-c5ca-44ee-9795-c260af5804bc None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:13.496291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82549ef9-c5ca-44ee-9795-c260af5804bc None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:13.496291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82549ef9-c5ca-44ee-9795-c260af5804bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:13.516383 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-82549ef9-c5ca-44ee-9795-c260af5804bc None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:38:13.517797 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-82549ef9-c5ca-44ee-9795-c260af5804bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:38:13.526509 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 690/1370] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:38:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:38:13.571798 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aef7cf-bde8-45ad-9576-900589bbd485 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:38:13.586722 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aef7cf-bde8-45ad-9576-900589bbd485 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:38:13.592331 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5377b562-966f-4789-99b7-33e21b4bb218 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:38:13.593131 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377b562-966f-4789-99b7-33e21b4bb218 service nova] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:13.593521 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377b562-966f-4789-99b7-33e21b4bb218 service nova] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:13.593905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377b562-966f-4789-99b7-33e21b4bb218 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:13.603009 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5377b562-966f-4789-99b7-33e21b4bb218 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:38:13.604707 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5377b562-966f-4789-99b7-33e21b4bb218 service nova] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:38:13.615939 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 681/1371] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:38:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:38:14.211093 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead3b41c-aabf-4b12-aff5-69f2443720da None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:14.211651 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead3b41c-aabf-4b12-aff5-69f2443720da None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:14.212151 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead3b41c-aabf-4b12-aff5-69f2443720da None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:14.214420 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 691/1372] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:38:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:38:14.219828 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0a9b0-213f-4a12-9fa3-7ea3cbb6f125 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:14.220126 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0a9b0-213f-4a12-9fa3-7ea3cbb6f125 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:14.220410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0a9b0-213f-4a12-9fa3-7ea3cbb6f125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:14.239375 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-ace0a9b0-213f-4a12-9fa3-7ea3cbb6f125 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:38:14.240858 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-ace0a9b0-213f-4a12-9fa3-7ea3cbb6f125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:38:14.265525 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 682/1373] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:38:14.609285 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26f13c-8c4d-46ad-bd3f-bfef9658ef24 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:14.609784 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26f13c-8c4d-46ad-bd3f-bfef9658ef24 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:14.609879 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26f13c-8c4d-46ad-bd3f-bfef9658ef24 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:14.611213 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 692/1374] 10.222.0.20 () {54 vars in 963 bytes} [Wed Jul 6 13:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:38:14.617106 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8592a-adf5-4361-97bc-f69ae0720a39 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:14.617607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8592a-adf5-4361-97bc-f69ae0720a39 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:14.617704 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8592a-adf5-4361-97bc-f69ae0720a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:14.669300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-21e8592a-adf5-4361-97bc-f69ae0720a39 None None] MFA Rules not processed for user `5997058ea0fe456a89856e222722a5cc`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:38:14.670606 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-21e8592a-adf5-4361-97bc-f69ae0720a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:38:14.737165 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 683/1375] 10.222.0.20 () {58 vars in 1060 bytes} [Wed Jul 6 13:38:14 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:38:14.742427 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26f13c-8c4d-46ad-bd3f-bfef9658ef24 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:38:14.755582 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26f13c-8c4d-46ad-bd3f-bfef9658ef24 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:38:14.759666 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e3b02-595c-4c54-94f5-27af17cd6dcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:38:14.760324 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e3b02-595c-4c54-94f5-27af17cd6dcf service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:14.760602 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e3b02-595c-4c54-94f5-27af17cd6dcf service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:14.761114 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e3b02-595c-4c54-94f5-27af17cd6dcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:14.771519 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e3b02-595c-4c54-94f5-27af17cd6dcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:38:14.773131 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e3b02-595c-4c54-94f5-27af17cd6dcf service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:38:14.784107 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 693/1376] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:38:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:38:15.728580 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692e8ab-f3b5-40bc-ad32-40692c6d08b3 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:15.729114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692e8ab-f3b5-40bc-ad32-40692c6d08b3 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:15.729267 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692e8ab-f3b5-40bc-ad32-40692c6d08b3 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:15.730724 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 684/1377] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:38:15.736091 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd564c-6b1c-4513-beb4-2c688f921659 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:15.736499 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd564c-6b1c-4513-beb4-2c688f921659 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:15.736680 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd564c-6b1c-4513-beb4-2c688f921659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:15.754388 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-fafd564c-6b1c-4513-beb4-2c688f921659 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:38:15.755562 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-fafd564c-6b1c-4513-beb4-2c688f921659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:38:15.763292 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 694/1378] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:38:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:38:15.807951 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692e8ab-f3b5-40bc-ad32-40692c6d08b3 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:38:15.822452 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692e8ab-f3b5-40bc-ad32-40692c6d08b3 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:38:15.827991 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bffd92-3f0e-4846-8450-144315456ac5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:38:15.828743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bffd92-3f0e-4846-8450-144315456ac5 service nova] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:15.829050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bffd92-3f0e-4846-8450-144315456ac5 service nova] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:15.829353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bffd92-3f0e-4846-8450-144315456ac5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:15.838096 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bffd92-3f0e-4846-8450-144315456ac5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:38:15.839690 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bffd92-3f0e-4846-8450-144315456ac5 service nova] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:38:15.843480 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:38:15.850635 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 685/1379] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:38:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:38:17.935556 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48e45a-a0a1-4f80-af0e-dde350454d92 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:17.936036 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48e45a-a0a1-4f80-af0e-dde350454d92 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:17.936036 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48e45a-a0a1-4f80-af0e-dde350454d92 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:17.937447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 695/1380] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:38:17.943255 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5056f3-ef92-43fd-9c1f-11863af8afb3 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:17.943674 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5056f3-ef92-43fd-9c1f-11863af8afb3 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:17.943805 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5056f3-ef92-43fd-9c1f-11863af8afb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:17.961375 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-3a5056f3-ef92-43fd-9c1f-11863af8afb3 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:38:17.962567 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-3a5056f3-ef92-43fd-9c1f-11863af8afb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:38:17.970322 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 686/1381] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:38:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:38:18.015259 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48e45a-a0a1-4f80-af0e-dde350454d92 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:38:18.029583 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48e45a-a0a1-4f80-af0e-dde350454d92 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:38:18.035036 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23bbeb5-b1ea-4331-83d6-bc1ee443a6af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:38:18.035711 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23bbeb5-b1ea-4331-83d6-bc1ee443a6af service nova] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:18.035994 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23bbeb5-b1ea-4331-83d6-bc1ee443a6af service nova] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:18.036303 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23bbeb5-b1ea-4331-83d6-bc1ee443a6af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:18.044448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23bbeb5-b1ea-4331-83d6-bc1ee443a6af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:38:18.046043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23bbeb5-b1ea-4331-83d6-bc1ee443a6af service nova] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:38:18.063786 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 696/1382] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:38:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:38:20.171867 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0305a-589c-4c70-acfc-23db87aa5288 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:20.172316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0305a-589c-4c70-acfc-23db87aa5288 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:20.172435 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0305a-589c-4c70-acfc-23db87aa5288 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:20.174077 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 687/1383] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:38:20.180846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b33d38-8602-4f15-8764-c5fcdc005d23 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:20.181316 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b33d38-8602-4f15-8764-c5fcdc005d23 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:20.181452 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b33d38-8602-4f15-8764-c5fcdc005d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:20.199996 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-f0b33d38-8602-4f15-8764-c5fcdc005d23 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:38:20.201523 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-f0b33d38-8602-4f15-8764-c5fcdc005d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:38:20.217586 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 697/1384] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:38:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:38:20.255718 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0305a-589c-4c70-acfc-23db87aa5288 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:38:20.271120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0305a-589c-4c70-acfc-23db87aa5288 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:38:20.276429 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5683f9eb-8d76-42d6-bd90-79d6b3cc81a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:38:20.277115 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683f9eb-8d76-42d6-bd90-79d6b3cc81a7 service nova] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:20.277402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683f9eb-8d76-42d6-bd90-79d6b3cc81a7 service nova] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:20.277682 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683f9eb-8d76-42d6-bd90-79d6b3cc81a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:20.287303 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5683f9eb-8d76-42d6-bd90-79d6b3cc81a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:38:20.288871 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5683f9eb-8d76-42d6-bd90-79d6b3cc81a7 service nova] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:38:20.309335 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 688/1385] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:38:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:38:22.397264 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfda5d-ec3b-4190-b1de-4c6d7a7039c0 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:22.397809 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfda5d-ec3b-4190-b1de-4c6d7a7039c0 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:22.397809 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfda5d-ec3b-4190-b1de-4c6d7a7039c0 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:22.399263 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 698/1386] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:38:22.404398 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef9750-578e-4cc6-ade0-c9c234eb9962 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:22.404847 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef9750-578e-4cc6-ade0-c9c234eb9962 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:22.405138 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef9750-578e-4cc6-ade0-c9c234eb9962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:22.423522 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-86ef9750-578e-4cc6-ade0-c9c234eb9962 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:38:22.425419 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-86ef9750-578e-4cc6-ade0-c9c234eb9962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:38:22.434494 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 689/1387] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:38:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:38:22.478319 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfda5d-ec3b-4190-b1de-4c6d7a7039c0 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:38:22.493705 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfda5d-ec3b-4190-b1de-4c6d7a7039c0 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:38:22.499661 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781719a4-2d4c-4a85-925b-206f7a2ba94b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:38:22.500326 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781719a4-2d4c-4a85-925b-206f7a2ba94b service nova] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:22.500639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781719a4-2d4c-4a85-925b-206f7a2ba94b service nova] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:22.500979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781719a4-2d4c-4a85-925b-206f7a2ba94b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:22.511224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781719a4-2d4c-4a85-925b-206f7a2ba94b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:38:22.512865 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781719a4-2d4c-4a85-925b-206f7a2ba94b service nova] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:38:22.522789 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 699/1388] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:38:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:38:24.685959 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7fdafa-3787-402b-a12b-ab67ddc03fd4 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:24.686650 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7fdafa-3787-402b-a12b-ab67ddc03fd4 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:24.686650 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7fdafa-3787-402b-a12b-ab67ddc03fd4 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:24.688216 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 690/1389] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:38:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:38:24.694236 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872df3ef-42a0-450b-8d4e-b0502e0746dd None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:24.694689 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872df3ef-42a0-450b-8d4e-b0502e0746dd None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:24.695011 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872df3ef-42a0-450b-8d4e-b0502e0746dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:24.717482 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-872df3ef-42a0-450b-8d4e-b0502e0746dd None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:38:24.718963 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-872df3ef-42a0-450b-8d4e-b0502e0746dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:38:24.729724 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 700/1390] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:38:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:38:24.776054 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7fdafa-3787-402b-a12b-ab67ddc03fd4 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:38:24.794182 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7fdafa-3787-402b-a12b-ab67ddc03fd4 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:38:24.800382 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9356d-1910-493c-9137-8a4457f3819a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:38:24.801050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9356d-1910-493c-9137-8a4457f3819a service nova] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:24.801341 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9356d-1910-493c-9137-8a4457f3819a service nova] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:24.801619 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9356d-1910-493c-9137-8a4457f3819a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:24.810107 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc9356d-1910-493c-9137-8a4457f3819a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:38:24.811693 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc9356d-1910-493c-9137-8a4457f3819a service nova] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:38:24.821950 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 691/1391] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:38:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:38:26.992743 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c4b2e-7b45-4163-ab97-6fb32d59a438 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:26.993439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c4b2e-7b45-4163-ab97-6fb32d59a438 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:26.993439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c4b2e-7b45-4163-ab97-6fb32d59a438 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:26.994730 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 701/1392] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:38:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:38:27.000957 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2cf7d1-5399-4ffb-9d14-34a452d71b4f None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:27.001426 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2cf7d1-5399-4ffb-9d14-34a452d71b4f None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:27.001584 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2cf7d1-5399-4ffb-9d14-34a452d71b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:27.019929 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-bd2cf7d1-5399-4ffb-9d14-34a452d71b4f None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:38:27.021168 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-bd2cf7d1-5399-4ffb-9d14-34a452d71b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:38:27.029349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 692/1393] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:38:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:38:27.086790 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c4b2e-7b45-4163-ab97-6fb32d59a438 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:38:27.101812 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c4b2e-7b45-4163-ab97-6fb32d59a438 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:38:27.107296 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0c2c9-4fac-4d5e-8a8f-2bc577771812 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:38:27.107934 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0c2c9-4fac-4d5e-8a8f-2bc577771812 service nova] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:27.108225 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0c2c9-4fac-4d5e-8a8f-2bc577771812 service nova] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:27.108507 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0c2c9-4fac-4d5e-8a8f-2bc577771812 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:27.117056 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b0c2c9-4fac-4d5e-8a8f-2bc577771812 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:38:27.118643 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b0c2c9-4fac-4d5e-8a8f-2bc577771812 service nova] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:38:27.128274 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 702/1394] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:38:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:38:29.282666 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc053f8-58a7-40ce-a5a4-4139c32c560f None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:29.283234 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc053f8-58a7-40ce-a5a4-4139c32c560f None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:29.283234 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc053f8-58a7-40ce-a5a4-4139c32c560f None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:29.285259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 693/1395] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:38:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:38:29.290420 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe4990-b7d9-4302-ae80-0c331061267a None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:29.290884 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe4990-b7d9-4302-ae80-0c331061267a None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:29.290982 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe4990-b7d9-4302-ae80-0c331061267a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:29.309477 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-3bbe4990-b7d9-4302-ae80-0c331061267a None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:38:29.310669 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-3bbe4990-b7d9-4302-ae80-0c331061267a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:38:29.318722 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 703/1396] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:38:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:38:29.363457 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc053f8-58a7-40ce-a5a4-4139c32c560f None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:38:29.377659 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc053f8-58a7-40ce-a5a4-4139c32c560f None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:38:29.382940 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64364bc-8475-4dc1-a5da-03f59748b456 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:38:29.383572 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64364bc-8475-4dc1-a5da-03f59748b456 service nova] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:29.383851 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64364bc-8475-4dc1-a5da-03f59748b456 service nova] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:29.384125 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64364bc-8475-4dc1-a5da-03f59748b456 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:29.392109 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64364bc-8475-4dc1-a5da-03f59748b456 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:38:29.394194 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64364bc-8475-4dc1-a5da-03f59748b456 service nova] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:38:29.403397 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 694/1397] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:38:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:38:29.962752 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffa016-8cc5-4a1f-8c98-4504b9f1cd00 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:29.963051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffa016-8cc5-4a1f-8c98-4504b9f1cd00 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:29.963298 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffa016-8cc5-4a1f-8c98-4504b9f1cd00 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:29.964788 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 704/1398] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:38:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:38:29.970409 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36999d7c-559d-456a-bf6d-16eae9a54a45 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:29.970942 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36999d7c-559d-456a-bf6d-16eae9a54a45 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:29.971165 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36999d7c-559d-456a-bf6d-16eae9a54a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:29.989898 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-36999d7c-559d-456a-bf6d-16eae9a54a45 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:38:29.991085 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-36999d7c-559d-456a-bf6d-16eae9a54a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:38:29.998881 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 695/1399] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:38:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:38:30.054579 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62107b-2663-43b1-81e1-8e919be4d676 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:30.054952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62107b-2663-43b1-81e1-8e919be4d676 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:30.055234 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62107b-2663-43b1-81e1-8e919be4d676 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:30.056835 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 705/1400] 10.222.0.20 () {54 vars in 962 bytes} [Wed Jul 6 13:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:38:30.062026 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac36092-cf6f-4133-8b92-efb96810c0b9 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:30.062388 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac36092-cf6f-4133-8b92-efb96810c0b9 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:30.062729 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac36092-cf6f-4133-8b92-efb96810c0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:30.112312 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-5ac36092-cf6f-4133-8b92-efb96810c0b9 None None] MFA Rules not processed for user `4015c5ca2b8a438288820cd003f9a308`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:38:30.113519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-5ac36092-cf6f-4133-8b92-efb96810c0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:38:30.172581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 696/1401] 10.222.0.20 () {58 vars in 1059 bytes} [Wed Jul 6 13:38:30 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:38:30.181332 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62107b-2663-43b1-81e1-8e919be4d676 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:38:30.192496 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62107b-2663-43b1-81e1-8e919be4d676 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:38:30.195964 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be6309c-573e-40ed-b554-2e8f99f55666 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4015c5ca2b8a438288820cd003f9a308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:38:30.196559 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be6309c-573e-40ed-b554-2e8f99f55666 service glance] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:30.196841 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be6309c-573e-40ed-b554-2e8f99f55666 service glance] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:30.197153 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be6309c-573e-40ed-b554-2e8f99f55666 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:30.205932 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be6309c-573e-40ed-b554-2e8f99f55666 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:38:30.207321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be6309c-573e-40ed-b554-2e8f99f55666 service glance] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:38:30.217182 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 706/1402] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:38:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:38:30.918062 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235c46a-28aa-4b06-b099-29baf3b8a565 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:30.918574 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235c46a-28aa-4b06-b099-29baf3b8a565 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:30.918664 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235c46a-28aa-4b06-b099-29baf3b8a565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:30.971661 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-b235c46a-28aa-4b06-b099-29baf3b8a565 None None] MFA Rules not processed for user `c4535134181e489fb532f617264d5855`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:38:30.972872 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-b235c46a-28aa-4b06-b099-29baf3b8a565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:38:31.014023 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:38:31.039677 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 697/1403] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Jul 6 13:38:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:38:31.085342 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e019da-388c-4867-898e-dc4a888998cb None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:31.085830 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e019da-388c-4867-898e-dc4a888998cb None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:31.085933 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e019da-388c-4867-898e-dc4a888998cb None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:31.087297 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 707/1404] 10.222.0.20 () {54 vars in 942 bytes} [Wed Jul 6 13:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:38:31.093635 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9cccf-b5e7-48ca-a2aa-2c7c50502071 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:31.093926 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9cccf-b5e7-48ca-a2aa-2c7c50502071 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:31.094205 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9cccf-b5e7-48ca-a2aa-2c7c50502071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:31.144101 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-8bb9cccf-b5e7-48ca-a2aa-2c7c50502071 None None] MFA Rules not processed for user `8c1bea05c37646b0a05fa8dda0367ef2`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:38:31.145199 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-8bb9cccf-b5e7-48ca-a2aa-2c7c50502071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:38:31.215637 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 698/1405] 10.222.0.20 () {58 vars in 1039 bytes} [Wed Jul 6 13:38:31 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:38:31.224737 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e019da-388c-4867-898e-dc4a888998cb None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:38:31.232179 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=61 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:38:31.239287 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e019da-388c-4867-898e-dc4a888998cb None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:38:31.244256 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c70fa-ed5b-4b1e-8ed8-98698d242274 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c1bea05c37646b0a05fa8dda0367ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:38:31.244982 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c70fa-ed5b-4b1e-8ed8-98698d242274 service swift] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:31.245370 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c70fa-ed5b-4b1e-8ed8-98698d242274 service swift] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:31.245722 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c70fa-ed5b-4b1e-8ed8-98698d242274 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:31.254840 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60c70fa-ed5b-4b1e-8ed8-98698d242274 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:38:31.256439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60c70fa-ed5b-4b1e-8ed8-98698d242274 service swift] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:38:31.264738 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 708/1406] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jul 6 13:38:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 13:38:31.499360 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b5254-cf54-42e1-b2fe-032862ee8d5e None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:31.499924 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b5254-cf54-42e1-b2fe-032862ee8d5e None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:31.500214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b5254-cf54-42e1-b2fe-032862ee8d5e None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:31.501626 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 699/1407] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:38:31.507824 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97211277-8445-4539-a226-d9b5f9ec6c66 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:31.508376 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97211277-8445-4539-a226-d9b5f9ec6c66 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:31.508657 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97211277-8445-4539-a226-d9b5f9ec6c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:31.527244 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-97211277-8445-4539-a226-d9b5f9ec6c66 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:38:31.528884 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-97211277-8445-4539-a226-d9b5f9ec6c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:38:31.537176 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 709/1408] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:38:31.582980 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b5254-cf54-42e1-b2fe-032862ee8d5e None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:38:31.598461 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b5254-cf54-42e1-b2fe-032862ee8d5e None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:38:31.603915 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d574bd4a-7627-461b-8387-777dd702e2c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:38:31.604559 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574bd4a-7627-461b-8387-777dd702e2c7 service nova] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:31.604844 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574bd4a-7627-461b-8387-777dd702e2c7 service nova] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:31.605154 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574bd4a-7627-461b-8387-777dd702e2c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:31.614047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d574bd4a-7627-461b-8387-777dd702e2c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:38:31.615626 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d574bd4a-7627-461b-8387-777dd702e2c7 service nova] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:38:31.626313 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 700/1409] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:38:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:38:41.521762 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=8 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:38:41.621692 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=59 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:38:53.625356 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5708313-5bd6-4dda-b730-ac9f0dcba997 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:53.625903 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5708313-5bd6-4dda-b730-ac9f0dcba997 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:53.625903 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5708313-5bd6-4dda-b730-ac9f0dcba997 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:53.627624 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 710/1410] 10.222.0.20 () {54 vars in 918 bytes} [Wed Jul 6 13:38:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:38:53.633359 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5592c88-10b5-46e5-addb-5dc09a04f612 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:53.633897 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5592c88-10b5-46e5-addb-5dc09a04f612 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:53.634075 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5592c88-10b5-46e5-addb-5dc09a04f612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:53.653281 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-e5592c88-10b5-46e5-addb-5dc09a04f612 None None] MFA Rules not processed for user `890d2ae150d647a18dae25159cf97484`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:38:53.654472 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-e5592c88-10b5-46e5-addb-5dc09a04f612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:38:53.663649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 701/1411] 10.222.0.20 () {58 vars in 1015 bytes} [Wed Jul 6 13:38:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:38:53.678469 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4c0a1-737c-40c2-b654-6811dbce4049 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:53.678769 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4c0a1-737c-40c2-b654-6811dbce4049 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:53.679051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4c0a1-737c-40c2-b654-6811dbce4049 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:53.680410 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 711/1412] 10.222.0.20 () {54 vars in 918 bytes} [Wed Jul 6 13:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:38:53.685049 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cecf7-5458-4881-91a7-bd71eeee356a None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:53.685343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cecf7-5458-4881-91a7-bd71eeee356a None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:53.685622 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cecf7-5458-4881-91a7-bd71eeee356a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:53.703652 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-426cecf7-5458-4881-91a7-bd71eeee356a None None] MFA Rules not processed for user `6647eb1e8a1544ef9d3a9806f2d1fc51`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:38:53.704825 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-426cecf7-5458-4881-91a7-bd71eeee356a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:38:53.714036 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 702/1413] 10.222.0.20 () {58 vars in 1015 bytes} [Wed Jul 6 13:38:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:38:56.226181 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a9b0e-f287-4191-9058-fd0cc1eab744 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:56.226733 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a9b0e-f287-4191-9058-fd0cc1eab744 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:56.226733 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a9b0e-f287-4191-9058-fd0cc1eab744 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:56.228074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 13:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:38:56.233297 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6ba35-acbf-4e40-9a07-58bc6e027886 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:56.233672 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6ba35-acbf-4e40-9a07-58bc6e027886 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:56.234028 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6ba35-acbf-4e40-9a07-58bc6e027886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:56.253074 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-4cb6ba35-acbf-4e40-9a07-58bc6e027886 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:38:56.254348 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-4cb6ba35-acbf-4e40-9a07-58bc6e027886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:38:56.262037 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 13:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:38:56.304654 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a9b0e-f287-4191-9058-fd0cc1eab744 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:38:56.320597 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a9b0e-f287-4191-9058-fd0cc1eab744 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:38:56.326189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfa0cb-169f-4a26-8cc8-2ae70c734ef7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:38:56.326852 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfa0cb-169f-4a26-8cc8-2ae70c734ef7 service nova] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:56.327120 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfa0cb-169f-4a26-8cc8-2ae70c734ef7 service nova] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:56.327393 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfa0cb-169f-4a26-8cc8-2ae70c734ef7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:56.336615 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dfa0cb-169f-4a26-8cc8-2ae70c734ef7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:38:56.338213 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dfa0cb-169f-4a26-8cc8-2ae70c734ef7 service nova] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:38:56.349630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 713/1416] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:38:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:38:57.309719 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c79b5-4bcb-4855-a8dd-8a350b076092 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:57.310208 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c79b5-4bcb-4855-a8dd-8a350b076092 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:57.310336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c79b5-4bcb-4855-a8dd-8a350b076092 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:57.311675 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 13:38:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:38:57.317840 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e741c-ee94-423d-9a53-7240cf40c958 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:57.318345 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e741c-ee94-423d-9a53-7240cf40c958 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:57.318463 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e741c-ee94-423d-9a53-7240cf40c958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:57.341589 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-df0e741c-ee94-423d-9a53-7240cf40c958 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:38:57.342786 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-df0e741c-ee94-423d-9a53-7240cf40c958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:38:57.350753 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 13:38:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:38:57.501335 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c79b5-4bcb-4855-a8dd-8a350b076092 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:38:57.515730 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c79b5-4bcb-4855-a8dd-8a350b076092 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:38:57.521677 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d71e94-edb7-4b88-921c-6d7bd23d3196 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:38:57.522303 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d71e94-edb7-4b88-921c-6d7bd23d3196 service nova] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:57.522599 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d71e94-edb7-4b88-921c-6d7bd23d3196 service nova] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:57.522852 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d71e94-edb7-4b88-921c-6d7bd23d3196 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:57.530765 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d71e94-edb7-4b88-921c-6d7bd23d3196 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:38:57.532363 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d71e94-edb7-4b88-921c-6d7bd23d3196 service nova] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:38:57.541549 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 705/1419] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:38:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:38:58.425385 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7511e-f60c-47ae-b9b1-a1caee85da5e None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:58.425914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7511e-f60c-47ae-b9b1-a1caee85da5e None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:58.425914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7511e-f60c-47ae-b9b1-a1caee85da5e None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:58.427288 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 13:38:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:38:58.432185 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac9c6d-2c31-443b-8ca7-c36c20453d2f None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:58.432566 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac9c6d-2c31-443b-8ca7-c36c20453d2f None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:58.432841 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac9c6d-2c31-443b-8ca7-c36c20453d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:58.450674 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-38ac9c6d-2c31-443b-8ca7-c36c20453d2f None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:38:58.451860 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-38ac9c6d-2c31-443b-8ca7-c36c20453d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:38:58.460015 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 13:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:38:58.610070 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7511e-f60c-47ae-b9b1-a1caee85da5e None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:38:58.624197 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7511e-f60c-47ae-b9b1-a1caee85da5e None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:38:58.629814 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24715d4c-6f4d-4298-87c5-7fd1cc57810a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:38:58.630454 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24715d4c-6f4d-4298-87c5-7fd1cc57810a service nova] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:58.630739 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24715d4c-6f4d-4298-87c5-7fd1cc57810a service nova] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:58.631018 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24715d4c-6f4d-4298-87c5-7fd1cc57810a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:58.639046 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24715d4c-6f4d-4298-87c5-7fd1cc57810a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:38:58.640617 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24715d4c-6f4d-4298-87c5-7fd1cc57810a service nova] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:38:58.650594 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 716/1422] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:38:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 13:38:59.501392 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2994a8-0418-4918-be77-8ce6f9351ebb None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:59.501958 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2994a8-0418-4918-be77-8ce6f9351ebb None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:59.501958 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2994a8-0418-4918-be77-8ce6f9351ebb None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:59.503332 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 13:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:38:59.508786 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8dc424-cb5b-47f3-b229-cb67aa15b9cd None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:59.509217 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8dc424-cb5b-47f3-b229-cb67aa15b9cd None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:59.509403 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8dc424-cb5b-47f3-b229-cb67aa15b9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:59.526838 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-9e8dc424-cb5b-47f3-b229-cb67aa15b9cd None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:38:59.528041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-9e8dc424-cb5b-47f3-b229-cb67aa15b9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:38:59.536465 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 13:38:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:38:59.695936 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2994a8-0418-4918-be77-8ce6f9351ebb None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:38:59.710269 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2994a8-0418-4918-be77-8ce6f9351ebb None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:38:59.715511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e3bf3-23e3-4945-8a52-d46a800d0c49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:38:59.716138 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e3bf3-23e3-4945-8a52-d46a800d0c49 service nova] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:38:59.716420 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e3bf3-23e3-4945-8a52-d46a800d0c49 service nova] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:38:59.716690 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e3bf3-23e3-4945-8a52-d46a800d0c49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:38:59.725204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e3bf3-23e3-4945-8a52-d46a800d0c49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:38:59.726763 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e3bf3-23e3-4945-8a52-d46a800d0c49 service nova] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:38:59.735815 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 708/1425] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:38:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:39:00.591665 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fecde-4d06-4189-86b7-f9f08af1027a None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:00.592207 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fecde-4d06-4189-86b7-f9f08af1027a None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:00.592207 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fecde-4d06-4189-86b7-f9f08af1027a None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:00.593645 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 13:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:39:00.598625 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3795d-afb3-4069-b5e6-d249cdb20303 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:00.599071 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3795d-afb3-4069-b5e6-d249cdb20303 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:00.599179 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3795d-afb3-4069-b5e6-d249cdb20303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:00.617963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-44f3795d-afb3-4069-b5e6-d249cdb20303 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:39:00.619462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-44f3795d-afb3-4069-b5e6-d249cdb20303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:39:00.628373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 13:39:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:39:00.784273 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0fecde-4d06-4189-86b7-f9f08af1027a None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:39:00.798148 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0fecde-4d06-4189-86b7-f9f08af1027a None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:39:00.804457 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ba309b-a640-44af-9392-9f53f4d4769b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:39:00.805123 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba309b-a640-44af-9392-9f53f4d4769b service nova] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:00.805409 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba309b-a640-44af-9392-9f53f4d4769b service nova] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:00.805685 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba309b-a640-44af-9392-9f53f4d4769b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:00.813569 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ba309b-a640-44af-9392-9f53f4d4769b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:39:00.816120 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ba309b-a640-44af-9392-9f53f4d4769b service nova] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:39:00.825260 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 719/1428] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:39:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:39:01.632808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fa39b-6a53-41e6-b00a-5b7c4299d99e None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:01.633391 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fa39b-6a53-41e6-b00a-5b7c4299d99e None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:01.633391 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fa39b-6a53-41e6-b00a-5b7c4299d99e None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:01.634676 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 13:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:39:01.639636 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e40d7a-b39d-4c4e-87c8-dbafce3d9c31 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:01.640033 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e40d7a-b39d-4c4e-87c8-dbafce3d9c31 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:01.640207 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e40d7a-b39d-4c4e-87c8-dbafce3d9c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:01.658658 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-a4e40d7a-b39d-4c4e-87c8-dbafce3d9c31 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:39:01.659850 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-a4e40d7a-b39d-4c4e-87c8-dbafce3d9c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:39:01.667627 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 13:39:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:39:01.818860 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39fa39b-6a53-41e6-b00a-5b7c4299d99e None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:39:01.832888 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39fa39b-6a53-41e6-b00a-5b7c4299d99e None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:39:01.838191 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3d770-fbe4-461a-99a2-61e524e0acc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:39:01.838846 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3d770-fbe4-461a-99a2-61e524e0acc9 service nova] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:01.839129 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3d770-fbe4-461a-99a2-61e524e0acc9 service nova] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:01.839405 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3d770-fbe4-461a-99a2-61e524e0acc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:01.847938 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d3d770-fbe4-461a-99a2-61e524e0acc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:39:01.849541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d3d770-fbe4-461a-99a2-61e524e0acc9 service nova] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:39:01.858748 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 711/1431] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:39:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:39:02.657179 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc820ea-9ebc-42f5-a844-6d6b298528cf None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:02.657752 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc820ea-9ebc-42f5-a844-6d6b298528cf None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:02.657752 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc820ea-9ebc-42f5-a844-6d6b298528cf None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:02.659152 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 13:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:39:02.663947 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea44c39-4fa5-416b-b6d7-d7b6802e9a82 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:02.664346 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea44c39-4fa5-416b-b6d7-d7b6802e9a82 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:02.664512 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea44c39-4fa5-416b-b6d7-d7b6802e9a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:02.682557 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-2ea44c39-4fa5-416b-b6d7-d7b6802e9a82 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:39:02.683755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-2ea44c39-4fa5-416b-b6d7-d7b6802e9a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:39:02.692131 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 13:39:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:39:02.840025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc820ea-9ebc-42f5-a844-6d6b298528cf None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:39:02.854187 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc820ea-9ebc-42f5-a844-6d6b298528cf None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:39:02.859483 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db2787c-841f-4c61-bbc2-9dd96ec5920d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:39:02.860115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2787c-841f-4c61-bbc2-9dd96ec5920d service nova] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:02.860397 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2787c-841f-4c61-bbc2-9dd96ec5920d service nova] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:02.860672 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2787c-841f-4c61-bbc2-9dd96ec5920d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:02.868842 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db2787c-841f-4c61-bbc2-9dd96ec5920d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:39:02.870425 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db2787c-841f-4c61-bbc2-9dd96ec5920d service nova] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:39:02.880175 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 722/1434] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:39:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:39:03.675432 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7169a5-a8fa-4933-be25-fc6f24e55683 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:03.675958 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7169a5-a8fa-4933-be25-fc6f24e55683 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:03.675958 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7169a5-a8fa-4933-be25-fc6f24e55683 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:03.677299 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 13:39:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:39:03.682256 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb52ac-d242-4722-8646-6c4824a30b96 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:03.682667 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb52ac-d242-4722-8646-6c4824a30b96 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:03.682811 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb52ac-d242-4722-8646-6c4824a30b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:03.700740 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-17fb52ac-d242-4722-8646-6c4824a30b96 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:39:03.701942 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-17fb52ac-d242-4722-8646-6c4824a30b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:39:03.709782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 13:39:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:39:03.858600 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7169a5-a8fa-4933-be25-fc6f24e55683 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:39:03.872884 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7169a5-a8fa-4933-be25-fc6f24e55683 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:39:03.878915 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f2405-3f04-4204-a9d3-4d8f63e7737e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:39:03.879658 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f2405-3f04-4204-a9d3-4d8f63e7737e service nova] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:03.880042 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f2405-3f04-4204-a9d3-4d8f63e7737e service nova] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:03.880421 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f2405-3f04-4204-a9d3-4d8f63e7737e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:03.888795 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f2405-3f04-4204-a9d3-4d8f63e7737e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:39:03.890401 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f2405-3f04-4204-a9d3-4d8f63e7737e service nova] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:39:03.899594 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 714/1437] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:39:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:39:04.695942 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05a998-dde6-4e36-ba46-a8325eb1640d None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:04.696497 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05a998-dde6-4e36-ba46-a8325eb1640d None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:04.696497 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05a998-dde6-4e36-ba46-a8325eb1640d None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:04.697886 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 13:39:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:39:04.702634 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83755a67-5839-45ef-a169-562e7acd416c None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:04.703053 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83755a67-5839-45ef-a169-562e7acd416c None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:04.703197 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83755a67-5839-45ef-a169-562e7acd416c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:04.720796 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-83755a67-5839-45ef-a169-562e7acd416c None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:39:04.722099 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-83755a67-5839-45ef-a169-562e7acd416c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:39:04.729874 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 13:39:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:39:04.885749 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05a998-dde6-4e36-ba46-a8325eb1640d None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:39:04.899802 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05a998-dde6-4e36-ba46-a8325eb1640d None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:39:04.905154 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565671d8-193e-4f52-822e-d54fc7210b2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:39:04.905788 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565671d8-193e-4f52-822e-d54fc7210b2b service nova] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:04.906073 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565671d8-193e-4f52-822e-d54fc7210b2b service nova] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:04.906350 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565671d8-193e-4f52-822e-d54fc7210b2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:04.914863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565671d8-193e-4f52-822e-d54fc7210b2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:39:04.916428 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565671d8-193e-4f52-822e-d54fc7210b2b service nova] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:39:04.926475 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 725/1440] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:39:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:39:06.900159 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded4676-4bb9-445e-9f40-eb6cf9927d94 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:06.900745 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded4676-4bb9-445e-9f40-eb6cf9927d94 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:06.900745 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded4676-4bb9-445e-9f40-eb6cf9927d94 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:06.902128 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 13:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:39:06.907557 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49e409-8038-4e2c-9873-deefd50ac319 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:06.909114 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49e409-8038-4e2c-9873-deefd50ac319 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:06.909114 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49e409-8038-4e2c-9873-deefd50ac319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:06.926580 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-1a49e409-8038-4e2c-9873-deefd50ac319 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:39:06.927803 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-1a49e409-8038-4e2c-9873-deefd50ac319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:39:06.935502 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 13:39:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:39:06.974733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded4676-4bb9-445e-9f40-eb6cf9927d94 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:39:06.987082 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded4676-4bb9-445e-9f40-eb6cf9927d94 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:39:06.991000 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a494ed-0695-411a-bea7-351b4370a041 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:39:06.991632 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a494ed-0695-411a-bea7-351b4370a041 service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:06.991914 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a494ed-0695-411a-bea7-351b4370a041 service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:06.992190 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a494ed-0695-411a-bea7-351b4370a041 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:07.000968 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a494ed-0695-411a-bea7-351b4370a041 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:39:07.002486 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a494ed-0695-411a-bea7-351b4370a041 service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:39:07.011619 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 717/1443] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:39:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:39:09.295706 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d1000-05a8-4d76-8503-61b434a60180 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:09.296174 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d1000-05a8-4d76-8503-61b434a60180 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:09.296282 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d1000-05a8-4d76-8503-61b434a60180 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:09.297863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 13:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:39:09.304524 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb37147-9e6a-4c8f-9551-3c6f26419786 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:09.304974 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb37147-9e6a-4c8f-9551-3c6f26419786 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:09.305122 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb37147-9e6a-4c8f-9551-3c6f26419786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:09.322637 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-cdb37147-9e6a-4c8f-9551-3c6f26419786 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:39:09.323827 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-cdb37147-9e6a-4c8f-9551-3c6f26419786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:39:09.331571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 13:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:39:09.369307 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d1000-05a8-4d76-8503-61b434a60180 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:39:09.379925 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d1000-05a8-4d76-8503-61b434a60180 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:39:09.383425 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af3411-c3da-4819-a95f-7c5a0828b1c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:39:09.384060 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af3411-c3da-4819-a95f-7c5a0828b1c4 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:09.384344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af3411-c3da-4819-a95f-7c5a0828b1c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:09.384622 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af3411-c3da-4819-a95f-7c5a0828b1c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:09.393516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af3411-c3da-4819-a95f-7c5a0828b1c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:39:09.396465 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af3411-c3da-4819-a95f-7c5a0828b1c4 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:39:09.406161 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 728/1446] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:39:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:39:11.778745 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ee135-2cd7-4ff4-a1a0-1d8ddac9b4f1 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:11.779345 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ee135-2cd7-4ff4-a1a0-1d8ddac9b4f1 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:11.779345 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ee135-2cd7-4ff4-a1a0-1d8ddac9b4f1 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:11.780712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 13:39:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:39:11.786122 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acc7db-fcc1-467e-a322-367b0019971b None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:11.786523 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acc7db-fcc1-467e-a322-367b0019971b None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:11.786678 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acc7db-fcc1-467e-a322-367b0019971b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:11.804327 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-c3acc7db-fcc1-467e-a322-367b0019971b None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:39:11.805552 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-c3acc7db-fcc1-467e-a322-367b0019971b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:39:11.813210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 13:39:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:39:11.851507 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ee135-2cd7-4ff4-a1a0-1d8ddac9b4f1 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:39:11.861933 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ee135-2cd7-4ff4-a1a0-1d8ddac9b4f1 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:39:11.865361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0547ba8-6ebd-40e5-ba79-c8644191c162 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:39:11.865986 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0547ba8-6ebd-40e5-ba79-c8644191c162 service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:11.866268 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0547ba8-6ebd-40e5-ba79-c8644191c162 service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:11.866537 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0547ba8-6ebd-40e5-ba79-c8644191c162 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:11.875105 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0547ba8-6ebd-40e5-ba79-c8644191c162 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:39:11.877237 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0547ba8-6ebd-40e5-ba79-c8644191c162 service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:39:11.886953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 720/1449] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:39:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:39:14.066079 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd109751-b6fb-41a7-808d-2c54bbc2d2a0 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:14.066648 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd109751-b6fb-41a7-808d-2c54bbc2d2a0 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:14.066648 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd109751-b6fb-41a7-808d-2c54bbc2d2a0 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:14.068038 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 13:39:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:39:14.073422 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68034d53-e954-4be0-be72-4640b1f1e6f4 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:14.073815 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68034d53-e954-4be0-be72-4640b1f1e6f4 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:14.073986 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68034d53-e954-4be0-be72-4640b1f1e6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:14.091498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-68034d53-e954-4be0-be72-4640b1f1e6f4 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:39:14.092691 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-68034d53-e954-4be0-be72-4640b1f1e6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:39:14.101024 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 13:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:39:14.138796 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd109751-b6fb-41a7-808d-2c54bbc2d2a0 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:39:14.150242 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd109751-b6fb-41a7-808d-2c54bbc2d2a0 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:39:14.153818 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b22bb6-6855-41cc-bd5e-72586db13183 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:39:14.154452 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b22bb6-6855-41cc-bd5e-72586db13183 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:14.154738 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b22bb6-6855-41cc-bd5e-72586db13183 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:14.155016 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b22bb6-6855-41cc-bd5e-72586db13183 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:14.165065 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b22bb6-6855-41cc-bd5e-72586db13183 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:39:14.166477 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b22bb6-6855-41cc-bd5e-72586db13183 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:39:14.176895 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 731/1452] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:39:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:39:16.280518 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829b403-7869-4363-abff-d0687348683e None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:16.281079 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829b403-7869-4363-abff-d0687348683e None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:16.281362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829b403-7869-4363-abff-d0687348683e None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:16.283240 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 13:39:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:39:16.288659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037b1d0-8f99-4aff-9e9c-224762b74293 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:16.289069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037b1d0-8f99-4aff-9e9c-224762b74293 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:16.289261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037b1d0-8f99-4aff-9e9c-224762b74293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:16.308200 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-f037b1d0-8f99-4aff-9e9c-224762b74293 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:39:16.309447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-f037b1d0-8f99-4aff-9e9c-224762b74293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:39:16.318737 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 13:39:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:39:16.357325 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829b403-7869-4363-abff-d0687348683e None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:39:16.368378 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829b403-7869-4363-abff-d0687348683e None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:39:16.371829 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728a012-d853-461c-9344-95cb8430b191 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:39:16.372456 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728a012-d853-461c-9344-95cb8430b191 service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:16.372735 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728a012-d853-461c-9344-95cb8430b191 service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:16.373024 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728a012-d853-461c-9344-95cb8430b191 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:16.381917 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7728a012-d853-461c-9344-95cb8430b191 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:39:16.383614 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7728a012-d853-461c-9344-95cb8430b191 service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:39:16.392863 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 723/1455] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:39:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:39:18.443647 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491f004-a5bb-4215-83d6-25809045afb7 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:18.444244 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491f004-a5bb-4215-83d6-25809045afb7 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:18.444244 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491f004-a5bb-4215-83d6-25809045afb7 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:18.445591 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 13:39:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:39:18.450811 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa496c5-3c6f-4039-a123-193e1391350b None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:18.451264 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa496c5-3c6f-4039-a123-193e1391350b None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:18.451392 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa496c5-3c6f-4039-a123-193e1391350b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:18.468912 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-2aa496c5-3c6f-4039-a123-193e1391350b None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:39:18.470657 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-2aa496c5-3c6f-4039-a123-193e1391350b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:39:18.478559 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 13:39:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:39:18.515689 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491f004-a5bb-4215-83d6-25809045afb7 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:39:18.526284 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491f004-a5bb-4215-83d6-25809045afb7 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:39:18.529778 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d128e7d7-284f-4ee7-8fd4-456890a40d5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:39:18.530407 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128e7d7-284f-4ee7-8fd4-456890a40d5c service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:18.530691 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128e7d7-284f-4ee7-8fd4-456890a40d5c service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:18.530967 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128e7d7-284f-4ee7-8fd4-456890a40d5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:18.540145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d128e7d7-284f-4ee7-8fd4-456890a40d5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:39:18.541624 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d128e7d7-284f-4ee7-8fd4-456890a40d5c service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:39:18.551647 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 734/1458] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:39:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:39:20.680672 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4798b32-afed-46e6-ad68-4b7f396454dd None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:20.681213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4798b32-afed-46e6-ad68-4b7f396454dd None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:20.681213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4798b32-afed-46e6-ad68-4b7f396454dd None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:20.682528 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 13:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:39:20.687595 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efeabbc-cf4a-4e96-8ebd-a903dab8c677 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:20.688006 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efeabbc-cf4a-4e96-8ebd-a903dab8c677 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:20.688188 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efeabbc-cf4a-4e96-8ebd-a903dab8c677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:20.707236 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-1efeabbc-cf4a-4e96-8ebd-a903dab8c677 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:39:20.708416 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-1efeabbc-cf4a-4e96-8ebd-a903dab8c677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:39:20.716114 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 13:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:39:20.753391 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4798b32-afed-46e6-ad68-4b7f396454dd None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:39:20.763840 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4798b32-afed-46e6-ad68-4b7f396454dd None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:39:20.767976 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3c1ad5-aa6b-4ef1-a9a4-55f85a4e11e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:39:20.768598 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c1ad5-aa6b-4ef1-a9a4-55f85a4e11e6 service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:20.768880 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c1ad5-aa6b-4ef1-a9a4-55f85a4e11e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:20.769192 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c1ad5-aa6b-4ef1-a9a4-55f85a4e11e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:20.777826 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3c1ad5-aa6b-4ef1-a9a4-55f85a4e11e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:39:20.779243 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3c1ad5-aa6b-4ef1-a9a4-55f85a4e11e6 service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:39:20.788396 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 726/1461] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:39:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 13:39:22.878830 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6faa1a-d9c4-45a8-ba0c-75e8b0ae49eb None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:22.879371 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6faa1a-d9c4-45a8-ba0c-75e8b0ae49eb None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:22.879371 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6faa1a-d9c4-45a8-ba0c-75e8b0ae49eb None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:22.880771 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 13:39:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:39:22.885961 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0a392-635b-4a66-9fc1-cc92c73a3066 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:22.886361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0a392-635b-4a66-9fc1-cc92c73a3066 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:22.886527 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0a392-635b-4a66-9fc1-cc92c73a3066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:22.893394 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=79 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:39:22.906018 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-b8b0a392-635b-4a66-9fc1-cc92c73a3066 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:39:22.907218 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-b8b0a392-635b-4a66-9fc1-cc92c73a3066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:39:22.915039 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 13:39:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:39:22.952382 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6faa1a-d9c4-45a8-ba0c-75e8b0ae49eb None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:39:22.958541 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=73 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:39:22.965129 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6faa1a-d9c4-45a8-ba0c-75e8b0ae49eb None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:39:22.968617 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204babbb-997d-4f71-8aed-a86322564f01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:39:22.969287 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204babbb-997d-4f71-8aed-a86322564f01 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:22.969575 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204babbb-997d-4f71-8aed-a86322564f01 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:22.969856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204babbb-997d-4f71-8aed-a86322564f01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:22.979115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204babbb-997d-4f71-8aed-a86322564f01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:39:22.980708 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204babbb-997d-4f71-8aed-a86322564f01 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:39:22.990625 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 737/1464] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:39:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:39:25.023647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699be31-cc1b-4e43-8571-25f174f54ebf None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:25.024181 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699be31-cc1b-4e43-8571-25f174f54ebf None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:25.024181 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699be31-cc1b-4e43-8571-25f174f54ebf None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:25.025591 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 13:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:39:25.030695 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33302cc4-770b-4a6a-885c-3d63fc700db4 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:25.031101 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33302cc4-770b-4a6a-885c-3d63fc700db4 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:25.031288 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33302cc4-770b-4a6a-885c-3d63fc700db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:25.050919 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-33302cc4-770b-4a6a-885c-3d63fc700db4 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:39:25.052106 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-33302cc4-770b-4a6a-885c-3d63fc700db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:39:25.059789 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 13:39:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:39:25.094871 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699be31-cc1b-4e43-8571-25f174f54ebf None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:39:25.105928 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699be31-cc1b-4e43-8571-25f174f54ebf None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:39:25.109345 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bc1146-50c4-48f3-b414-511422f895cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:39:25.109970 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc1146-50c4-48f3-b414-511422f895cc service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:25.110252 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc1146-50c4-48f3-b414-511422f895cc service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:25.110713 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc1146-50c4-48f3-b414-511422f895cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:25.119759 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bc1146-50c4-48f3-b414-511422f895cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:39:25.121200 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bc1146-50c4-48f3-b414-511422f895cc service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:39:25.130286 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 729/1467] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:39:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:39:27.173558 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b86386-1b87-4e78-89a9-23405ba939fe None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:27.174092 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b86386-1b87-4e78-89a9-23405ba939fe None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:27.174092 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b86386-1b87-4e78-89a9-23405ba939fe None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:27.175750 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 13:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:39:27.181034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebebdf1-4d93-457d-b68c-c564ad350994 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:27.181438 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebebdf1-4d93-457d-b68c-c564ad350994 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:27.181586 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebebdf1-4d93-457d-b68c-c564ad350994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:27.198822 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-cebebdf1-4d93-457d-b68c-c564ad350994 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:39:27.199924 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-cebebdf1-4d93-457d-b68c-c564ad350994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:39:27.208153 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 13:39:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:39:27.243779 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b86386-1b87-4e78-89a9-23405ba939fe None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:39:27.254277 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b86386-1b87-4e78-89a9-23405ba939fe None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:39:27.257724 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086bb2e8-3987-4d52-9e22-b2c93d10659c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:39:27.258350 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bb2e8-3987-4d52-9e22-b2c93d10659c service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:27.258631 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bb2e8-3987-4d52-9e22-b2c93d10659c service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:27.258932 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bb2e8-3987-4d52-9e22-b2c93d10659c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:27.268133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086bb2e8-3987-4d52-9e22-b2c93d10659c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:39:27.269581 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086bb2e8-3987-4d52-9e22-b2c93d10659c service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:39:27.278659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 740/1470] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:39:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:39:37.193888 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=11 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:39:37.274590 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=14 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:39:47.146275 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc36af-6947-495c-b60b-5b126412f9df None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:47.146815 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc36af-6947-495c-b60b-5b126412f9df None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:47.146815 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc36af-6947-495c-b60b-5b126412f9df None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:47.148247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 731/1471] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:39:47.153415 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6d121-c3f4-4130-b834-b100c408902f None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:47.153858 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6d121-c3f4-4130-b834-b100c408902f None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:47.153975 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6d121-c3f4-4130-b834-b100c408902f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:47.171023 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-21e6d121-c3f4-4130-b834-b100c408902f None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:39:47.172189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-21e6d121-c3f4-4130-b834-b100c408902f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:39:47.180689 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 741/1472] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:39:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:39:47.263774 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f603275-dfc3-40ec-97bb-eb1a600b2243 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:47.264059 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f603275-dfc3-40ec-97bb-eb1a600b2243 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:47.264337 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f603275-dfc3-40ec-97bb-eb1a600b2243 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:47.265755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 732/1473] 10.222.0.20 () {54 vars in 962 bytes} [Wed Jul 6 13:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:39:47.270595 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036534ea-a54b-491f-999d-d0d33e9161ba None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:47.270887 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036534ea-a54b-491f-999d-d0d33e9161ba None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:47.271170 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036534ea-a54b-491f-999d-d0d33e9161ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:47.289493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-036534ea-a54b-491f-999d-d0d33e9161ba None None] MFA Rules not processed for user `4015c5ca2b8a438288820cd003f9a308`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:39:47.290696 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-036534ea-a54b-491f-999d-d0d33e9161ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:39:47.297224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 742/1474] 10.222.0.20 () {58 vars in 1059 bytes} [Wed Jul 6 13:39:47 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:39:47.305520 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f603275-dfc3-40ec-97bb-eb1a600b2243 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:39:47.316623 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f603275-dfc3-40ec-97bb-eb1a600b2243 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:39:47.320025 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bb0b86-5ff6-4d7e-babe-a0579d119f6d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4015c5ca2b8a438288820cd003f9a308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:39:47.320647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb0b86-5ff6-4d7e-babe-a0579d119f6d service glance] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:47.320927 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb0b86-5ff6-4d7e-babe-a0579d119f6d service glance] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:47.321235 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb0b86-5ff6-4d7e-babe-a0579d119f6d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:47.330038 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bb0b86-5ff6-4d7e-babe-a0579d119f6d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:39:47.331446 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bb0b86-5ff6-4d7e-babe-a0579d119f6d service glance] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:39:47.340706 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 733/1475] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:39:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:39:49.458005 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c970f-d5f8-4e0a-9ee0-fa00171ff219 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:49.458583 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c970f-d5f8-4e0a-9ee0-fa00171ff219 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:49.458583 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c970f-d5f8-4e0a-9ee0-fa00171ff219 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:49.459947 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 743/1476] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:39:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:39:49.465130 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93a4ee-2276-4dcb-9324-49d464395604 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:49.465537 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93a4ee-2276-4dcb-9324-49d464395604 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:49.465695 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93a4ee-2276-4dcb-9324-49d464395604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:49.483627 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-4e93a4ee-2276-4dcb-9324-49d464395604 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:39:49.484844 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-4e93a4ee-2276-4dcb-9324-49d464395604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:39:49.492872 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 734/1477] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:39:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:39:49.516936 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c970f-d5f8-4e0a-9ee0-fa00171ff219 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:39:49.529194 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c970f-d5f8-4e0a-9ee0-fa00171ff219 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:39:49.534248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c85e8f-d66d-4a53-abf0-1f351434972f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:39:49.534887 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c85e8f-d66d-4a53-abf0-1f351434972f admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:49.535174 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c85e8f-d66d-4a53-abf0-1f351434972f admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:49.535456 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c85e8f-d66d-4a53-abf0-1f351434972f admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:49.536191 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c85e8f-d66d-4a53-abf0-1f351434972f admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:39:49.538004 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: WARNING py.warnings [None req-b3c85e8f-d66d-4a53-abf0-1f351434972f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 13:39:49.538004 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: warnings.warn(msg) Jul 06 13:39:49.538004 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: Jul 06 13:39:49.539138 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c85e8f-d66d-4a53-abf0-1f351434972f admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:39:49.558399 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 744/1478] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jul 6 13:39:49 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:39:51.621493 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8b8d7-fe84-4563-900a-5c1476fe4945 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:51.622017 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8b8d7-fe84-4563-900a-5c1476fe4945 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:51.622017 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8b8d7-fe84-4563-900a-5c1476fe4945 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:51.623325 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 735/1479] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:39:51.628483 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aadc3bd-5009-420f-aac4-2bd1316922a8 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:51.628910 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aadc3bd-5009-420f-aac4-2bd1316922a8 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:51.629083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aadc3bd-5009-420f-aac4-2bd1316922a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:51.646859 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-6aadc3bd-5009-420f-aac4-2bd1316922a8 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:39:51.648062 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-6aadc3bd-5009-420f-aac4-2bd1316922a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:39:51.655824 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 745/1480] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:39:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:39:51.699213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8b8d7-fe84-4563-900a-5c1476fe4945 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:39:51.713750 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8b8d7-fe84-4563-900a-5c1476fe4945 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:39:51.718988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ac9ef-5be6-4dc9-bec0-6b41b18cc0d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:39:51.720315 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ac9ef-5be6-4dc9-bec0-6b41b18cc0d3 service nova] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:51.720743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ac9ef-5be6-4dc9-bec0-6b41b18cc0d3 service nova] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:51.721043 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ac9ef-5be6-4dc9-bec0-6b41b18cc0d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:51.729212 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ac9ef-5be6-4dc9-bec0-6b41b18cc0d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:39:51.730779 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ac9ef-5be6-4dc9-bec0-6b41b18cc0d3 service nova] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:39:51.739814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 736/1481] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:39:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:39:53.962573 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18197a0-c4b8-415d-903b-2e0af721dbc1 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:53.963168 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18197a0-c4b8-415d-903b-2e0af721dbc1 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:53.963168 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18197a0-c4b8-415d-903b-2e0af721dbc1 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:53.964511 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 746/1482] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:39:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:39:53.970470 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075a383-5eb4-4437-9756-f03c404df650 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:53.970915 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075a383-5eb4-4437-9756-f03c404df650 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:53.971022 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075a383-5eb4-4437-9756-f03c404df650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:53.988990 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-9075a383-5eb4-4437-9756-f03c404df650 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:39:53.990191 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-9075a383-5eb4-4437-9756-f03c404df650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:39:53.997836 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 737/1483] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:39:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:39:54.033855 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18197a0-c4b8-415d-903b-2e0af721dbc1 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:39:54.044496 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18197a0-c4b8-415d-903b-2e0af721dbc1 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:39:54.048207 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23901670-df6d-4cd1-9ef4-8b39cd9a88fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:39:54.048833 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23901670-df6d-4cd1-9ef4-8b39cd9a88fd service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:54.049158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23901670-df6d-4cd1-9ef4-8b39cd9a88fd service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:54.049433 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23901670-df6d-4cd1-9ef4-8b39cd9a88fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:54.058295 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23901670-df6d-4cd1-9ef4-8b39cd9a88fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:39:54.059724 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23901670-df6d-4cd1-9ef4-8b39cd9a88fd service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:39:54.069073 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 747/1484] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:39:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:39:56.173418 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a49b88-73c2-4d6c-aee8-7893ef334f29 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:56.173985 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a49b88-73c2-4d6c-aee8-7893ef334f29 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:56.173985 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a49b88-73c2-4d6c-aee8-7893ef334f29 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:56.175602 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 738/1485] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:39:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:39:56.180817 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe21552-2c89-42d8-b537-8b62a43fc34b None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:56.181281 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe21552-2c89-42d8-b537-8b62a43fc34b None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:56.181522 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe21552-2c89-42d8-b537-8b62a43fc34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:56.199701 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-8fe21552-2c89-42d8-b537-8b62a43fc34b None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:39:56.200879 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-8fe21552-2c89-42d8-b537-8b62a43fc34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:39:56.208595 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 748/1486] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:39:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:39:56.248348 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a49b88-73c2-4d6c-aee8-7893ef334f29 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:39:56.259576 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a49b88-73c2-4d6c-aee8-7893ef334f29 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:39:56.263103 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e60d22-96f2-4a7e-9c21-4e786f8f4c30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:39:56.263656 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e60d22-96f2-4a7e-9c21-4e786f8f4c30 service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:56.264010 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e60d22-96f2-4a7e-9c21-4e786f8f4c30 service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:56.264191 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e60d22-96f2-4a7e-9c21-4e786f8f4c30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:56.272901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e60d22-96f2-4a7e-9c21-4e786f8f4c30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:39:56.274343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e60d22-96f2-4a7e-9c21-4e786f8f4c30 service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:39:56.283541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 739/1487] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:39:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:39:58.510568 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b94911-8bf0-4440-8056-3d55bb1d6aea None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:58.511122 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b94911-8bf0-4440-8056-3d55bb1d6aea None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:58.511122 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b94911-8bf0-4440-8056-3d55bb1d6aea None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:58.512559 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 749/1488] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:39:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:39:58.518225 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d084e8-5f41-4765-9500-6b70d21d5b2f None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:58.518671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d084e8-5f41-4765-9500-6b70d21d5b2f None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:58.518785 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d084e8-5f41-4765-9500-6b70d21d5b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:58.536649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-12d084e8-5f41-4765-9500-6b70d21d5b2f None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:39:58.537856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-12d084e8-5f41-4765-9500-6b70d21d5b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:39:58.545497 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 740/1489] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:39:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:39:58.585620 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b94911-8bf0-4440-8056-3d55bb1d6aea None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:39:58.596672 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b94911-8bf0-4440-8056-3d55bb1d6aea None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:39:58.600086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b605613b-1705-4dc1-ae6a-dc89f3be2de0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:39:58.600712 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605613b-1705-4dc1-ae6a-dc89f3be2de0 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:39:58.600992 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605613b-1705-4dc1-ae6a-dc89f3be2de0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:39:58.601302 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605613b-1705-4dc1-ae6a-dc89f3be2de0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:39:58.609924 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b605613b-1705-4dc1-ae6a-dc89f3be2de0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:39:58.611384 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b605613b-1705-4dc1-ae6a-dc89f3be2de0 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:39:58.620531 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 750/1490] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:39:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:40:00.821380 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3028f4-9e67-4562-93aa-5f2353ef49be None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:40:00.821937 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3028f4-9e67-4562-93aa-5f2353ef49be None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:40:00.821937 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3028f4-9e67-4562-93aa-5f2353ef49be None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:40:00.823287 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 741/1491] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jul 6 13:40:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:40:00.828514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365dfbc5-8759-4291-9cbf-3f3cda43d66e None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:40:00.828959 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365dfbc5-8759-4291-9cbf-3f3cda43d66e None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:40:00.829111 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365dfbc5-8759-4291-9cbf-3f3cda43d66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:40:00.847506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-365dfbc5-8759-4291-9cbf-3f3cda43d66e None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:40:00.848702 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-365dfbc5-8759-4291-9cbf-3f3cda43d66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:40:00.856408 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 751/1492] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jul 6 13:40:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:40:00.895316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3028f4-9e67-4562-93aa-5f2353ef49be None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:40:00.905959 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3028f4-9e67-4562-93aa-5f2353ef49be None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:40:00.909516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af7990-21b7-41f7-a7e1-49a75648d850 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:40:00.910158 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af7990-21b7-41f7-a7e1-49a75648d850 service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:40:00.910444 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af7990-21b7-41f7-a7e1-49a75648d850 service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:40:00.910723 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af7990-21b7-41f7-a7e1-49a75648d850 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:40:00.919471 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af7990-21b7-41f7-a7e1-49a75648d850 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:40:00.920901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af7990-21b7-41f7-a7e1-49a75648d850 service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:40:00.930972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 742/1493] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:40:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:40:10.841423 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=33 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:40:10.926845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=33 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:41:34.277996 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8dac3-b372-4903-b44a-66d4b96f05ff None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:41:34.278592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8dac3-b372-4903-b44a-66d4b96f05ff None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:41:34.278592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8dac3-b372-4903-b44a-66d4b96f05ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:41:34.298040 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-bdc8dac3-b372-4903-b44a-66d4b96f05ff None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:41:34.299245 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-bdc8dac3-b372-4903-b44a-66d4b96f05ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:41:34.307529 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 752/1494] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:41:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:41:34.315939 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c498a27-3cd7-47ea-8a4f-b9c4260298a6 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:41:34.316421 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c498a27-3cd7-47ea-8a4f-b9c4260298a6 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:41:34.316526 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c498a27-3cd7-47ea-8a4f-b9c4260298a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:41:34.334474 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-3c498a27-3cd7-47ea-8a4f-b9c4260298a6 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:41:34.335658 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-3c498a27-3cd7-47ea-8a4f-b9c4260298a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:41:34.344088 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 743/1495] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:41:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:41:34.348597 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8dac3-b372-4903-b44a-66d4b96f05ff None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:41:34.357956 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8dac3-b372-4903-b44a-66d4b96f05ff None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:41:34.363433 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff514b-9a7c-4f98-b30a-97fb1584ec33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:41:34.364186 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff514b-9a7c-4f98-b30a-97fb1584ec33 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:41:34.364472 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff514b-9a7c-4f98-b30a-97fb1584ec33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:41:34.364745 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff514b-9a7c-4f98-b30a-97fb1584ec33 admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:41:34.365526 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff514b-9a7c-4f98-b30a-97fb1584ec33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:41:34.365828 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff514b-9a7c-4f98-b30a-97fb1584ec33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:41:34.366831 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: WARNING py.warnings [None req-0dff514b-9a7c-4f98-b30a-97fb1584ec33 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 06 13:41:34.366831 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: warnings.warn(msg) Jul 06 13:41:34.366831 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: Jul 06 13:41:34.367705 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff514b-9a7c-4f98-b30a-97fb1584ec33 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:41:34.378904 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 753/1496] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:41:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:41:34.386725 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c498a27-3cd7-47ea-8a4f-b9c4260298a6 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:41:34.396176 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c498a27-3cd7-47ea-8a4f-b9c4260298a6 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:41:34.400615 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f1de1b-2cad-4564-b0f0-89765d6d4fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:41:34.401299 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1de1b-2cad-4564-b0f0-89765d6d4fe7 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:41:34.401580 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1de1b-2cad-4564-b0f0-89765d6d4fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:41:34.401852 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1de1b-2cad-4564-b0f0-89765d6d4fe7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:41:34.402528 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f1de1b-2cad-4564-b0f0-89765d6d4fe7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:41:34.404294 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: WARNING py.warnings [None req-49f1de1b-2cad-4564-b0f0-89765d6d4fe7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 13:41:34.404294 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: warnings.warn(msg) Jul 06 13:41:34.404294 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: Jul 06 13:41:34.406108 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f1de1b-2cad-4564-b0f0-89765d6d4fe7 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:41:34.443207 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 744/1497] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:41:34 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:41:34.447247 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff514b-9a7c-4f98-b30a-97fb1584ec33 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:41:34.457551 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff514b-9a7c-4f98-b30a-97fb1584ec33 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:41:34.510918 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20ffbf-de7b-447c-8f1a-cf02cc5043a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:41:34.511575 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20ffbf-de7b-447c-8f1a-cf02cc5043a6 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:41:34.511858 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20ffbf-de7b-447c-8f1a-cf02cc5043a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:41:34.512135 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20ffbf-de7b-447c-8f1a-cf02cc5043a6 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:41:34.512822 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d20ffbf-de7b-447c-8f1a-cf02cc5043a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:41:34.513856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: WARNING py.warnings [None req-0d20ffbf-de7b-447c-8f1a-cf02cc5043a6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 13:41:34.513856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: warnings.warn(msg) Jul 06 13:41:34.513856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: Jul 06 13:41:34.515536 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d20ffbf-de7b-447c-8f1a-cf02cc5043a6 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:41:34.536062 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 754/1498] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:41:34 2022] POST /identity/v3/users => generated 289 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:41:34.540066 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f1de1b-2cad-4564-b0f0-89765d6d4fe7 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:41:34.550210 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f1de1b-2cad-4564-b0f0-89765d6d4fe7 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:41:34.554997 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb78736-b842-446f-bec2-005ffda3f0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:41:34.555674 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb78736-b842-446f-bec2-005ffda3f0cb admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:41:34.555955 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb78736-b842-446f-bec2-005ffda3f0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:41:34.556281 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb78736-b842-446f-bec2-005ffda3f0cb admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:41:34.557023 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb78736-b842-446f-bec2-005ffda3f0cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:41:34.558023 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: WARNING py.warnings [None req-3bb78736-b842-446f-bec2-005ffda3f0cb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 06 13:41:34.558023 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: warnings.warn(msg) Jul 06 13:41:34.558023 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: Jul 06 13:41:34.558889 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb78736-b842-446f-bec2-005ffda3f0cb admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:41:34.571342 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 745/1499] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:41:34.575167 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20ffbf-de7b-447c-8f1a-cf02cc5043a6 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:41:34.585277 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20ffbf-de7b-447c-8f1a-cf02cc5043a6 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:41:34.589681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f059be-d8bd-47bc-877b-7310de3d9f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:41:34.590327 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f059be-d8bd-47bc-877b-7310de3d9f3b admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:41:34.590613 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f059be-d8bd-47bc-877b-7310de3d9f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:41:34.590888 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f059be-d8bd-47bc-877b-7310de3d9f3b admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:41:34.591605 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f059be-d8bd-47bc-877b-7310de3d9f3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:41:34.592595 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: WARNING py.warnings [None req-e0f059be-d8bd-47bc-877b-7310de3d9f3b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 06 13:41:34.592595 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: warnings.warn(msg) Jul 06 13:41:34.592595 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: Jul 06 13:41:34.594042 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f059be-d8bd-47bc-877b-7310de3d9f3b admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:41:34.607353 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 755/1500] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:41:34.611233 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb78736-b842-446f-bec2-005ffda3f0cb admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:41:34.620903 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb78736-b842-446f-bec2-005ffda3f0cb admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:41:34.625361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d59ee0-1c9e-4abf-a3e7-b22d89efdde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:41:34.626228 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d59ee0-1c9e-4abf-a3e7-b22d89efdde3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:41:34.626510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d59ee0-1c9e-4abf-a3e7-b22d89efdde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:41:34.626783 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d59ee0-1c9e-4abf-a3e7-b22d89efdde3 admin admin] PATH_INFO: `/v3/projects/293a848d03d0462aa73a5526194bc850/users/c29844f2dfba495797dd014c2eb22c04/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:41:34.641388 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d59ee0-1c9e-4abf-a3e7-b22d89efdde3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=293a848d03d0462aa73a5526194bc850, user_id=c29844f2dfba495797dd014c2eb22c04, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:41:34.642409 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: WARNING py.warnings [None req-a0d59ee0-1c9e-4abf-a3e7-b22d89efdde3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 13:41:34.642409 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: warnings.warn(msg) Jul 06 13:41:34.642409 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: Jul 06 13:41:34.644319 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d59ee0-1c9e-4abf-a3e7-b22d89efdde3 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:41:34.658806 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 746/1501] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:41:34 2022] PUT /identity/v3/projects/293a848d03d0462aa73a5526194bc850/users/c29844f2dfba495797dd014c2eb22c04/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:41:34.670825 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2e4f9-1884-4024-b542-cc363b179b28 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:41:34.671122 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2e4f9-1884-4024-b542-cc363b179b28 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:41:34.671413 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2e4f9-1884-4024-b542-cc363b179b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:41:34.696908 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-35b2e4f9-1884-4024-b542-cc363b179b28 None None] MFA Rules not processed for user `c29844f2dfba495797dd014c2eb22c04`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:41:34.698142 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-35b2e4f9-1884-4024-b542-cc363b179b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:41:34.762269 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 756/1502] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:41:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:41:34.775926 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a62d62-a674-48c7-8f51-5da428880150 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:41:34.776216 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a62d62-a674-48c7-8f51-5da428880150 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:41:34.776518 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a62d62-a674-48c7-8f51-5da428880150 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:41:34.777940 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 747/1503] 10.222.0.20 () {54 vars in 963 bytes} [Wed Jul 6 13:41:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:41:34.782764 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489b0ea-b763-41a5-a21b-3f228e566ffe None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:41:34.783061 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489b0ea-b763-41a5-a21b-3f228e566ffe None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:41:34.783345 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489b0ea-b763-41a5-a21b-3f228e566ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:41:34.801121 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-5489b0ea-b763-41a5-a21b-3f228e566ffe None None] MFA Rules not processed for user `5997058ea0fe456a89856e222722a5cc`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:41:34.802246 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-5489b0ea-b763-41a5-a21b-3f228e566ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:41:34.845552 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:41:34.854057 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 757/1504] 10.222.0.20 () {58 vars in 1060 bytes} [Wed Jul 6 13:41:34 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:41:34.861718 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a62d62-a674-48c7-8f51-5da428880150 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:41:34.871943 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a62d62-a674-48c7-8f51-5da428880150 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:41:34.876064 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffea7d6-b8bd-4c95-b37d-2e212aff6ea5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:41:34.876687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffea7d6-b8bd-4c95-b37d-2e212aff6ea5 service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:41:34.876965 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffea7d6-b8bd-4c95-b37d-2e212aff6ea5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:41:34.877265 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffea7d6-b8bd-4c95-b37d-2e212aff6ea5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:41:34.885518 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffea7d6-b8bd-4c95-b37d-2e212aff6ea5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:41:34.886957 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffea7d6-b8bd-4c95-b37d-2e212aff6ea5 service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:41:34.896819 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 748/1505] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:41:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:41:34.940741 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2b2bc-60f6-43c9-98b6-53d523619740 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:41:34.941065 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2b2bc-60f6-43c9-98b6-53d523619740 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:41:34.941347 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2b2bc-60f6-43c9-98b6-53d523619740 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:41:34.942623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 758/1506] 10.222.0.20 () {52 vars in 829 bytes} [Wed Jul 6 13:41:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:41:34.946234 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffea7d6-b8bd-4c95-b37d-2e212aff6ea5 service cinder] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:41:35.009677 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffea7d6-b8bd-4c95-b37d-2e212aff6ea5 service cinder] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:41:35.014114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b18544-d6fc-4d80-84e2-f6c3ee3e0cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:41:35.015574 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b18544-d6fc-4d80-84e2-f6c3ee3e0cdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:41:35.015807 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b18544-d6fc-4d80-84e2-f6c3ee3e0cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:41:35.016170 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b18544-d6fc-4d80-84e2-f6c3ee3e0cdc admin admin] PATH_INFO: `/v3/users/c29844f2dfba495797dd014c2eb22c04` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:41:35.017391 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b18544-d6fc-4d80-84e2-f6c3ee3e0cdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c29844f2dfba495797dd014c2eb22c04)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:41:35.018401 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: WARNING py.warnings [None req-32b18544-d6fc-4d80-84e2-f6c3ee3e0cdc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 13:41:35.018401 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: warnings.warn(msg) Jul 06 13:41:35.018401 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: Jul 06 13:41:35.019599 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b18544-d6fc-4d80-84e2-f6c3ee3e0cdc admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:41:35.083707 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-32b18544-d6fc-4d80-84e2-f6c3ee3e0cdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c29844f2dfba495797dd014c2eb22c04'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:41:35.093572 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-32b18544-d6fc-4d80-84e2-f6c3ee3e0cdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c29844f2dfba495797dd014c2eb22c04'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:41:35.094144 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-32b18544-d6fc-4d80-84e2-f6c3ee3e0cdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c29844f2dfba495797dd014c2eb22c04'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:41:35.105954 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-32b18544-d6fc-4d80-84e2-f6c3ee3e0cdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c29844f2dfba495797dd014c2eb22c04'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:41:35.122287 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:41:35.125948 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-32b18544-d6fc-4d80-84e2-f6c3ee3e0cdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c29844f2dfba495797dd014c2eb22c04'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:41:35.133496 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 749/1507] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:41:34 2022] DELETE /identity/v3/users/c29844f2dfba495797dd014c2eb22c04 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:41:35.137861 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66f9b7-fb8d-4b99-b873-f1443beede51 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:41:35.138151 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66f9b7-fb8d-4b99-b873-f1443beede51 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:41:35.138427 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66f9b7-fb8d-4b99-b873-f1443beede51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:41:35.155422 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-7c66f9b7-fb8d-4b99-b873-f1443beede51 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:41:35.156676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-7c66f9b7-fb8d-4b99-b873-f1443beede51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:41:35.213851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 759/1508] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:41:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:41:35.224297 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b18544-d6fc-4d80-84e2-f6c3ee3e0cdc admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:41:35.225709 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-32b18544-d6fc-4d80-84e2-f6c3ee3e0cdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:41:35.283196 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b18544-d6fc-4d80-84e2-f6c3ee3e0cdc admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:41:35.286770 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d16cbc-92db-4cb9-838f-6a22425ace19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:41:35.287417 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d16cbc-92db-4cb9-838f-6a22425ace19 service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:41:35.287706 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d16cbc-92db-4cb9-838f-6a22425ace19 service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:41:35.287987 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d16cbc-92db-4cb9-838f-6a22425ace19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:41:35.298063 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d16cbc-92db-4cb9-838f-6a22425ace19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:41:35.299517 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d16cbc-92db-4cb9-838f-6a22425ace19 service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:41:35.309241 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 750/1509] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:41:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:41:35.673373 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66f9b7-fb8d-4b99-b873-f1443beede51 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:41:35.674854 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-7c66f9b7-fb8d-4b99-b873-f1443beede51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:41:35.687858 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66f9b7-fb8d-4b99-b873-f1443beede51 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:41:35.692553 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a869267-41e8-496f-8a4c-0a7a4d76e8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:41:35.693477 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a869267-41e8-496f-8a4c-0a7a4d76e8c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:41:35.693762 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a869267-41e8-496f-8a4c-0a7a4d76e8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:41:35.694786 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a869267-41e8-496f-8a4c-0a7a4d76e8c3 admin admin] PATH_INFO: `/v3/projects/293a848d03d0462aa73a5526194bc850` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:41:35.696271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a869267-41e8-496f-8a4c-0a7a4d76e8c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=293a848d03d0462aa73a5526194bc850)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:41:35.697339 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: WARNING py.warnings [None req-8a869267-41e8-496f-8a4c-0a7a4d76e8c3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 13:41:35.697339 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: warnings.warn(msg) Jul 06 13:41:35.697339 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: Jul 06 13:41:35.699209 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a869267-41e8-496f-8a4c-0a7a4d76e8c3 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:41:35.769644 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8a869267-41e8-496f-8a4c-0a7a4d76e8c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '293a848d03d0462aa73a5526194bc850', 'request_id': 'req-8a869267-41e8-496f-8a4c-0a7a4d76e8c3'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:41:35.776145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8a869267-41e8-496f-8a4c-0a7a4d76e8c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '293a848d03d0462aa73a5526194bc850', 'request_id': 'req-8a869267-41e8-496f-8a4c-0a7a4d76e8c3'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:41:35.787144 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8a869267-41e8-496f-8a4c-0a7a4d76e8c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '293a848d03d0462aa73a5526194bc850', 'request_id': 'req-8a869267-41e8-496f-8a4c-0a7a4d76e8c3'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:41:35.809368 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8a869267-41e8-496f-8a4c-0a7a4d76e8c3 admin admin] The token cache is being invalidate because project 293a848d03d0462aa73a5526194bc850 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:41:35.809675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8a869267-41e8-496f-8a4c-0a7a4d76e8c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:41:35.810488 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8a869267-41e8-496f-8a4c-0a7a4d76e8c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:41:35.812707 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 760/1510] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:41:35 2022] DELETE /identity/v3/projects/293a848d03d0462aa73a5526194bc850 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:41:45.305058 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:41:45.805854 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:43:15.333244 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2e679-29c7-4a8f-ae38-ae8c3b6aa568 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:43:15.333784 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2e679-29c7-4a8f-ae38-ae8c3b6aa568 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:43:15.333900 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2e679-29c7-4a8f-ae38-ae8c3b6aa568 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:43:15.335239 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 13:43:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:43:15.339937 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdde2be-0995-4696-8f0c-284074378632 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:43:15.340346 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdde2be-0995-4696-8f0c-284074378632 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:43:15.340498 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdde2be-0995-4696-8f0c-284074378632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:43:15.358444 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-4fdde2be-0995-4696-8f0c-284074378632 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:43:15.359644 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-4fdde2be-0995-4696-8f0c-284074378632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:43:15.420149 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 13:43:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:43:15.576555 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2e679-29c7-4a8f-ae38-ae8c3b6aa568 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:43:15.578316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-79c2e679-29c7-4a8f-ae38-ae8c3b6aa568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:43:15.650346 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2e679-29c7-4a8f-ae38-ae8c3b6aa568 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:43:15.656189 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c242c10b-3cc9-44d3-b2b9-232cac9a46a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:43:15.656836 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242c10b-3cc9-44d3-b2b9-232cac9a46a2 service nova] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:43:15.657153 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242c10b-3cc9-44d3-b2b9-232cac9a46a2 service nova] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:43:15.657430 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242c10b-3cc9-44d3-b2b9-232cac9a46a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:43:15.666512 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c242c10b-3cc9-44d3-b2b9-232cac9a46a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:43:15.668180 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c242c10b-3cc9-44d3-b2b9-232cac9a46a2 service nova] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:43:15.678688 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 752/1513] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:43:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:43:16.515210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0657ce-bd24-46be-b9e8-a30a632c465b None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:43:16.515774 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0657ce-bd24-46be-b9e8-a30a632c465b None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:43:16.515774 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0657ce-bd24-46be-b9e8-a30a632c465b None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:43:16.517233 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 762/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 13:43:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:43:16.522001 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9571f-9102-444e-88c8-6eb5b20129ca None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:43:16.522431 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9571f-9102-444e-88c8-6eb5b20129ca None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:43:16.522544 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9571f-9102-444e-88c8-6eb5b20129ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:43:16.540571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-b1c9571f-9102-444e-88c8-6eb5b20129ca None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:43:16.541784 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-b1c9571f-9102-444e-88c8-6eb5b20129ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:43:16.549568 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 753/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 13:43:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:43:16.701425 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0657ce-bd24-46be-b9e8-a30a632c465b None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:43:16.716238 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0657ce-bd24-46be-b9e8-a30a632c465b None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:43:16.721631 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c039dd6e-f683-4c1f-ad01-267ebaadb34a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:43:16.722266 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c039dd6e-f683-4c1f-ad01-267ebaadb34a service nova] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:43:16.722552 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c039dd6e-f683-4c1f-ad01-267ebaadb34a service nova] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:43:16.722829 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c039dd6e-f683-4c1f-ad01-267ebaadb34a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:43:16.730828 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c039dd6e-f683-4c1f-ad01-267ebaadb34a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:43:16.732629 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c039dd6e-f683-4c1f-ad01-267ebaadb34a service nova] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:43:16.741932 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 763/1516] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:43:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:43:17.586024 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e26401-987c-4937-82c5-124d96de413f None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:43:17.586554 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e26401-987c-4937-82c5-124d96de413f None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:43:17.586554 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e26401-987c-4937-82c5-124d96de413f None None] PATH_INFO: `/v3` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:43:17.587796 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 754/1517] 10.222.0.39 () {54 vars in 925 bytes} [Wed Jul 6 13:43:17 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:43:17.605886 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd2292-71d2-483b-8ed2-ebb8e94c826a None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:43:17.606347 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd2292-71d2-483b-8ed2-ebb8e94c826a None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:43:17.606462 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd2292-71d2-483b-8ed2-ebb8e94c826a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:43:17.629506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-d8cd2292-71d2-483b-8ed2-ebb8e94c826a None None] MFA Rules not processed for user `890d2ae150d647a18dae25159cf97484`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:43:17.630956 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-d8cd2292-71d2-483b-8ed2-ebb8e94c826a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:43:17.699269 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 764/1518] 10.222.0.39 () {58 vars in 1015 bytes} [Wed Jul 6 13:43:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:43:26.535599 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=37 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:43:27.690854 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=62 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:43:27.967607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c886a-a8da-4c60-897e-0d694b04a4bb None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:43:27.968141 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c886a-a8da-4c60-897e-0d694b04a4bb None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:43:27.968141 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c886a-a8da-4c60-897e-0d694b04a4bb None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:43:27.969542 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 755/1519] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 13:43:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:43:27.974591 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e80068-b565-42ab-b284-57f11bc281f5 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:43:27.975017 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e80068-b565-42ab-b284-57f11bc281f5 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:43:27.975311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e80068-b565-42ab-b284-57f11bc281f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:43:27.978265 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9805f9-0c54-49d1-90f8-72fbfe48914c None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:43:27.978628 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9805f9-0c54-49d1-90f8-72fbfe48914c None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:43:27.978979 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9805f9-0c54-49d1-90f8-72fbfe48914c None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:43:27.980336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 756/1520] 10.222.0.20 () {54 vars in 960 bytes} [Wed Jul 6 13:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 13:43:27.985862 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdb996-a533-4a85-890e-22bab221ee95 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:43:27.986154 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdb996-a533-4a85-890e-22bab221ee95 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:43:27.986435 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdb996-a533-4a85-890e-22bab221ee95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:43:27.994049 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-60e80068-b565-42ab-b284-57f11bc281f5 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:43:27.995242 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-60e80068-b565-42ab-b284-57f11bc281f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:43:28.003390 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 765/1521] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 13:43:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:43:28.003931 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-ebbdb996-a533-4a85-890e-22bab221ee95 None None] MFA Rules not processed for user `890d2ae150d647a18dae25159cf97484`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:43:28.005145 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-ebbdb996-a533-4a85-890e-22bab221ee95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:43:28.014417 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 757/1522] 10.222.0.20 () {58 vars in 1057 bytes} [Wed Jul 6 13:43:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:43:28.022352 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e80068-b565-42ab-b284-57f11bc281f5 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:43:28.034245 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e80068-b565-42ab-b284-57f11bc281f5 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:43:28.039210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f1bee-fe88-4323-85d0-56ee00971ba7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '890d2ae150d647a18dae25159cf97484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:43:28.039942 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f1bee-fe88-4323-85d0-56ee00971ba7 service placement] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:43:28.040295 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f1bee-fe88-4323-85d0-56ee00971ba7 service placement] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:43:28.040647 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f1bee-fe88-4323-85d0-56ee00971ba7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:43:28.049873 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f1bee-fe88-4323-85d0-56ee00971ba7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:43:28.051788 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f1bee-fe88-4323-85d0-56ee00971ba7 service placement] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:43:28.061976 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 766/1523] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:43:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:43:28.165715 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbdb996-a533-4a85-890e-22bab221ee95 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:43:28.179292 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbdb996-a533-4a85-890e-22bab221ee95 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:43:28.184694 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc78a3b2-63ae-49c5-9b47-9f4f675758ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:43:28.185367 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78a3b2-63ae-49c5-9b47-9f4f675758ae service nova] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:43:28.185657 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78a3b2-63ae-49c5-9b47-9f4f675758ae service nova] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:43:28.185936 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78a3b2-63ae-49c5-9b47-9f4f675758ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:43:28.194358 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc78a3b2-63ae-49c5-9b47-9f4f675758ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:43:28.195949 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc78a3b2-63ae-49c5-9b47-9f4f675758ae service nova] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:43:28.205414 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 758/1524] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:43:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:43:29.076346 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2466cb-407b-4ab5-a356-02ea64b92384 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:43:29.076924 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2466cb-407b-4ab5-a356-02ea64b92384 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:43:29.076924 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2466cb-407b-4ab5-a356-02ea64b92384 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:43:29.078325 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 767/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 13:43:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:43:29.083176 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92b50d-0429-41cd-a03d-f52ca3261b32 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:43:29.083552 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92b50d-0429-41cd-a03d-f52ca3261b32 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:43:29.083739 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92b50d-0429-41cd-a03d-f52ca3261b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:43:29.103496 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-3c92b50d-0429-41cd-a03d-f52ca3261b32 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:43:29.104688 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-3c92b50d-0429-41cd-a03d-f52ca3261b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:43:29.114227 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 759/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 13:43:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:43:29.266807 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2466cb-407b-4ab5-a356-02ea64b92384 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:43:29.281645 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2466cb-407b-4ab5-a356-02ea64b92384 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:43:29.286809 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9717ec8-36e8-4a06-b836-83a0306e29a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:43:29.287435 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9717ec8-36e8-4a06-b836-83a0306e29a8 service nova] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:43:29.287716 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9717ec8-36e8-4a06-b836-83a0306e29a8 service nova] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:43:29.287990 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9717ec8-36e8-4a06-b836-83a0306e29a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:43:29.296768 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9717ec8-36e8-4a06-b836-83a0306e29a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:43:29.298412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9717ec8-36e8-4a06-b836-83a0306e29a8 service nova] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:43:29.308232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 768/1527] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:43:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:43:33.219946 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d81ac-3543-4eae-9914-efa8e1308a4e None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:43:33.220515 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d81ac-3543-4eae-9914-efa8e1308a4e None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:43:33.220515 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d81ac-3543-4eae-9914-efa8e1308a4e None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:43:33.221930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 760/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 13:43:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:43:33.227049 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c424125-65e5-4e5b-be2d-384d41482418 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:43:33.227489 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c424125-65e5-4e5b-be2d-384d41482418 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:43:33.227641 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c424125-65e5-4e5b-be2d-384d41482418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:43:33.245654 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-1c424125-65e5-4e5b-be2d-384d41482418 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:43:33.246847 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-1c424125-65e5-4e5b-be2d-384d41482418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:43:33.255528 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 769/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 13:43:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:43:33.331418 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d81ac-3543-4eae-9914-efa8e1308a4e None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:43:33.333145 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-904d81ac-3543-4eae-9914-efa8e1308a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:43:33.391010 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d81ac-3543-4eae-9914-efa8e1308a4e None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:43:33.394536 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac583987-3f6f-4afe-bf48-c61716254e50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:43:33.395186 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac583987-3f6f-4afe-bf48-c61716254e50 service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:43:33.395471 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac583987-3f6f-4afe-bf48-c61716254e50 service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:43:33.395749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac583987-3f6f-4afe-bf48-c61716254e50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:43:33.410468 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac583987-3f6f-4afe-bf48-c61716254e50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:43:33.411907 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac583987-3f6f-4afe-bf48-c61716254e50 service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:43:33.422820 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 761/1530] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:43:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:43:35.548271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337e2a1-1e64-430e-aacc-df4e889635d5 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:43:35.548863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337e2a1-1e64-430e-aacc-df4e889635d5 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:43:35.548863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337e2a1-1e64-430e-aacc-df4e889635d5 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:43:35.550167 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 770/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 13:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:43:35.555633 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebbd9a-f0b2-4230-b76a-e571cd744c5d None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:43:35.556099 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebbd9a-f0b2-4230-b76a-e571cd744c5d None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:43:35.556211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebbd9a-f0b2-4230-b76a-e571cd744c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:43:35.574855 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-2bebbd9a-f0b2-4230-b76a-e571cd744c5d None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:43:35.576723 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-2bebbd9a-f0b2-4230-b76a-e571cd744c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:43:35.585174 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 762/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 13:43:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:43:35.626250 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8337e2a1-1e64-430e-aacc-df4e889635d5 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:43:35.637397 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8337e2a1-1e64-430e-aacc-df4e889635d5 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:43:35.641528 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f9b50-7264-4c7b-9dcf-1f85cb5e67c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:43:35.642168 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f9b50-7264-4c7b-9dcf-1f85cb5e67c9 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:43:35.642455 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f9b50-7264-4c7b-9dcf-1f85cb5e67c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:43:35.642731 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f9b50-7264-4c7b-9dcf-1f85cb5e67c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:43:35.651779 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f9b50-7264-4c7b-9dcf-1f85cb5e67c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:43:35.653299 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f9b50-7264-4c7b-9dcf-1f85cb5e67c9 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:43:35.663448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 771/1533] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:43:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:43:39.531295 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56affa-69fa-4166-be80-351a181dddfd None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:43:39.531856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56affa-69fa-4166-be80-351a181dddfd None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:43:39.531856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56affa-69fa-4166-be80-351a181dddfd None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:43:39.533299 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 763/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 13:43:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:43:39.538383 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690fd4bc-fa44-4862-aa85-ca8cb7e96f88 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:43:39.538775 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690fd4bc-fa44-4862-aa85-ca8cb7e96f88 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:43:39.538947 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690fd4bc-fa44-4862-aa85-ca8cb7e96f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:43:39.558229 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-690fd4bc-fa44-4862-aa85-ca8cb7e96f88 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:43:39.559380 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-690fd4bc-fa44-4862-aa85-ca8cb7e96f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:43:39.567206 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 772/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 13:43:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:43:39.871082 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56affa-69fa-4166-be80-351a181dddfd None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:43:39.873068 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-7b56affa-69fa-4166-be80-351a181dddfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:43:39.929739 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56affa-69fa-4166-be80-351a181dddfd None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:43:39.933232 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dad73c-d6e4-43de-834c-4e23714801c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:43:39.933873 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dad73c-d6e4-43de-834c-4e23714801c5 service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:43:39.934156 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dad73c-d6e4-43de-834c-4e23714801c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:43:39.934812 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dad73c-d6e4-43de-834c-4e23714801c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:43:39.943181 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dad73c-d6e4-43de-834c-4e23714801c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:43:39.944603 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dad73c-d6e4-43de-834c-4e23714801c5 service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:43:39.954371 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 764/1536] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:43:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:43:42.085389 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee22cf7-6088-4549-b7ac-5e920a95b1cb None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:43:42.085961 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee22cf7-6088-4549-b7ac-5e920a95b1cb None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:43:42.085961 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee22cf7-6088-4549-b7ac-5e920a95b1cb None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:43:42.087605 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 773/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 13:43:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:43:42.093716 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49183c-6807-4ee4-b782-c82a753da820 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:43:42.094139 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49183c-6807-4ee4-b782-c82a753da820 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:43:42.094276 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49183c-6807-4ee4-b782-c82a753da820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:43:42.113738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-9b49183c-6807-4ee4-b782-c82a753da820 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:43:42.114933 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-9b49183c-6807-4ee4-b782-c82a753da820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:43:42.123824 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 765/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 13:43:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:43:42.464221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee22cf7-6088-4549-b7ac-5e920a95b1cb None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:43:42.466943 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-6ee22cf7-6088-4549-b7ac-5e920a95b1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:43:42.483320 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee22cf7-6088-4549-b7ac-5e920a95b1cb None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:43:42.486915 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3024e2-d090-42e5-86b7-817e74676bbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:43:42.487566 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3024e2-d090-42e5-86b7-817e74676bbd service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:43:42.487946 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3024e2-d090-42e5-86b7-817e74676bbd service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:43:42.488233 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3024e2-d090-42e5-86b7-817e74676bbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:43:42.498972 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3024e2-d090-42e5-86b7-817e74676bbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:43:42.500401 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3024e2-d090-42e5-86b7-817e74676bbd service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:43:42.510216 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 774/1539] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:43:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:43:52.108090 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=70 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:43:52.505706 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=33 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:12.698049 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e31142-12b8-45ba-97de-054993b0e936 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:12.698049 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e31142-12b8-45ba-97de-054993b0e936 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:12.698780 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e31142-12b8-45ba-97de-054993b0e936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:12.718374 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-98e31142-12b8-45ba-97de-054993b0e936 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:12.719584 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-98e31142-12b8-45ba-97de-054993b0e936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:12.736270 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 766/1540] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:12.745611 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1885b-9f45-43eb-a888-09a43a8dfea5 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:12.746211 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1885b-9f45-43eb-a888-09a43a8dfea5 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:12.746211 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1885b-9f45-43eb-a888-09a43a8dfea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:12.778876 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-bde1885b-9f45-43eb-a888-09a43a8dfea5 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:12.780120 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-bde1885b-9f45-43eb-a888-09a43a8dfea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:12.794482 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 775/1541] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:12.799425 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e31142-12b8-45ba-97de-054993b0e936 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:12.806750 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefeced-6556-467f-80d5-b48a4569ffce None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:12.807697 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefeced-6556-467f-80d5-b48a4569ffce None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:12.807829 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefeced-6556-467f-80d5-b48a4569ffce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:12.812087 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e31142-12b8-45ba-97de-054993b0e936 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:12.827032 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184bfa6a-3a96-4fdc-a8e4-aec06ecdd517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:12.827903 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184bfa6a-3a96-4fdc-a8e4-aec06ecdd517 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:12.828190 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184bfa6a-3a96-4fdc-a8e4-aec06ecdd517 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:12.828459 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184bfa6a-3a96-4fdc-a8e4-aec06ecdd517 admin admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:12.829609 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184bfa6a-3a96-4fdc-a8e4-aec06ecdd517 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:45:12.829609 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184bfa6a-3a96-4fdc-a8e4-aec06ecdd517 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:12.830570 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: WARNING py.warnings [None req-184bfa6a-3a96-4fdc-a8e4-aec06ecdd517 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 06 13:45:12.830570 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: warnings.warn(msg) Jul 06 13:45:12.830570 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: Jul 06 13:45:12.831450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184bfa6a-3a96-4fdc-a8e4-aec06ecdd517 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:12.839236 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-ebefeced-6556-467f-80d5-b48a4569ffce None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:12.840463 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-ebefeced-6556-467f-80d5-b48a4569ffce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:12.856440 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 767/1542] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:45:12.864866 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184bfa6a-3a96-4fdc-a8e4-aec06ecdd517 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:12.867173 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 776/1543] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 06 13:45:12.878496 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10a137-60fd-4a3d-8c79-4427f4e7b8ca None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:12.879081 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10a137-60fd-4a3d-8c79-4427f4e7b8ca None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:12.879680 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10a137-60fd-4a3d-8c79-4427f4e7b8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:12.893265 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184bfa6a-3a96-4fdc-a8e4-aec06ecdd517 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:12.903958 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24196d0b-e9d2-47ef-bd97-d0c37a7161f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:12.904805 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24196d0b-e9d2-47ef-bd97-d0c37a7161f6 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:12.905117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24196d0b-e9d2-47ef-bd97-d0c37a7161f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:12.905396 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24196d0b-e9d2-47ef-bd97-d0c37a7161f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:12.906094 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24196d0b-e9d2-47ef-bd97-d0c37a7161f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:12.908089 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24196d0b-e9d2-47ef-bd97-d0c37a7161f6 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:12.921934 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-2b10a137-60fd-4a3d-8c79-4427f4e7b8ca None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:12.923178 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-2b10a137-60fd-4a3d-8c79-4427f4e7b8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:12.950199 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 777/1544] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:12.955328 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfe125-0839-47b6-bb20-13507386a5fb None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:12.957035 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfe125-0839-47b6-bb20-13507386a5fb None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:12.957330 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfe125-0839-47b6-bb20-13507386a5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:12.970847 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 768/1545] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:45:12 2022] POST /identity/v3/projects => generated 379 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:45:12.971789 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24196d0b-e9d2-47ef-bd97-d0c37a7161f6 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:13.013244 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-63bfe125-0839-47b6-bb20-13507386a5fb None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:13.014469 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-63bfe125-0839-47b6-bb20-13507386a5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:13.113245 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24196d0b-e9d2-47ef-bd97-d0c37a7161f6 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:13.123051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 778/1546] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:13.123782 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6b8110-f06c-455e-805d-1f699e50f640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:13.123926 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bfe125-0839-47b6-bb20-13507386a5fb None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:13.124820 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b8110-f06c-455e-805d-1f699e50f640 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:13.125359 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b8110-f06c-455e-805d-1f699e50f640 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:13.126019 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b8110-f06c-455e-805d-1f699e50f640 admin admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:13.126976 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6b8110-f06c-455e-805d-1f699e50f640 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:45:13.127490 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6b8110-f06c-455e-805d-1f699e50f640 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:13.129721 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6b8110-f06c-455e-805d-1f699e50f640 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:13.140398 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bfe125-0839-47b6-bb20-13507386a5fb None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:13.145458 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f484416-7a8f-4093-acf8-6bc599af50ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:13.146136 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f484416-7a8f-4093-acf8-6bc599af50ff admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:13.146422 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f484416-7a8f-4093-acf8-6bc599af50ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:13.146701 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f484416-7a8f-4093-acf8-6bc599af50ff admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:13.147395 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f484416-7a8f-4093-acf8-6bc599af50ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:13.148443 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 769/1547] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:45:13.150767 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d6bb5c-5bac-488f-b946-f90713dddc78 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:13.151413 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d6bb5c-5bac-488f-b946-f90713dddc78 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:13.151925 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d6bb5c-5bac-488f-b946-f90713dddc78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:13.155948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f484416-7a8f-4093-acf8-6bc599af50ff admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:13.174635 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-d9d6bb5c-5bac-488f-b946-f90713dddc78 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:13.176598 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-d9d6bb5c-5bac-488f-b946-f90713dddc78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:13.187569 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 770/1548] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:13.193050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61113148-8483-4194-ac0d-6d9d60288aa3 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:13.193050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61113148-8483-4194-ac0d-6d9d60288aa3 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:13.193050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61113148-8483-4194-ac0d-6d9d60288aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:13.193562 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 779/1549] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:45:13 2022] POST /identity/v3/users => generated 290 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:13.195046 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3674b8c-1d2a-4fad-9141-38a75e052213 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:13.195354 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3674b8c-1d2a-4fad-9141-38a75e052213 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:13.195648 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3674b8c-1d2a-4fad-9141-38a75e052213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:13.221045 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-61113148-8483-4194-ac0d-6d9d60288aa3 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:13.221045 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-61113148-8483-4194-ac0d-6d9d60288aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:13.221359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-c3674b8c-1d2a-4fad-9141-38a75e052213 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:13.221359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-c3674b8c-1d2a-4fad-9141-38a75e052213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:13.226792 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 780/1550] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:13.228052 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc847b6-f23d-4626-81ef-f70d0f83cea9 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:13.228648 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc847b6-f23d-4626-81ef-f70d0f83cea9 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:13.228648 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc847b6-f23d-4626-81ef-f70d0f83cea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:13.229192 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 771/1551] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:13.230152 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61113148-8483-4194-ac0d-6d9d60288aa3 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:13.240156 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61113148-8483-4194-ac0d-6d9d60288aa3 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:13.244755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b371d333-6e69-43c4-bb62-a19b1fa61a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:13.245434 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371d333-6e69-43c4-bb62-a19b1fa61a02 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:13.245715 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371d333-6e69-43c4-bb62-a19b1fa61a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:13.246064 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371d333-6e69-43c4-bb62-a19b1fa61a02 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:13.246752 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b371d333-6e69-43c4-bb62-a19b1fa61a02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:13.248639 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b371d333-6e69-43c4-bb62-a19b1fa61a02 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:13.250738 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-0bc847b6-f23d-4626-81ef-f70d0f83cea9 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:13.251928 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-0bc847b6-f23d-4626-81ef-f70d0f83cea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:13.260569 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 781/1552] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:13.261975 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ad9ae-8134-41cb-b80b-42d26a8fb6df None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:13.262271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ad9ae-8134-41cb-b80b-42d26a8fb6df None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:13.262554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ad9ae-8134-41cb-b80b-42d26a8fb6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:13.281877 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 772/1553] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:45:13 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:13.283008 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b371d333-6e69-43c4-bb62-a19b1fa61a02 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:13.283134 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-1a0ad9ae-8134-41cb-b80b-42d26a8fb6df None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:13.284263 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-1a0ad9ae-8134-41cb-b80b-42d26a8fb6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:13.348515 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 782/1554] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:13.349839 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b805b-5dd8-4226-9bc8-a0a9119e83a8 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:13.350135 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b805b-5dd8-4226-9bc8-a0a9119e83a8 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:13.350411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b805b-5dd8-4226-9bc8-a0a9119e83a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:13.351329 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b371d333-6e69-43c4-bb62-a19b1fa61a02 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:13.356202 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454e1f5-7af6-48d1-b28c-0f1bfaf28405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:13.356992 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454e1f5-7af6-48d1-b28c-0f1bfaf28405 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:13.357340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454e1f5-7af6-48d1-b28c-0f1bfaf28405 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:13.357648 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454e1f5-7af6-48d1-b28c-0f1bfaf28405 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:13.358403 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7454e1f5-7af6-48d1-b28c-0f1bfaf28405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:13.360050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7454e1f5-7af6-48d1-b28c-0f1bfaf28405 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:13.368302 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-eb4b805b-5dd8-4226-9bc8-a0a9119e83a8 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:13.369697 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-eb4b805b-5dd8-4226-9bc8-a0a9119e83a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:13.373614 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 773/1555] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:13.373614 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a56471-f54f-4527-b2db-707efa5d2b03 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:13.373614 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a56471-f54f-4527-b2db-707efa5d2b03 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:13.373614 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a56471-f54f-4527-b2db-707efa5d2b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:13.377948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 783/1556] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:13.379237 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4657d9-de1c-4a26-8378-6920d9a96a20 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:13.379536 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4657d9-de1c-4a26-8378-6920d9a96a20 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:13.379818 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4657d9-de1c-4a26-8378-6920d9a96a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:13.392639 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-97a56471-f54f-4527-b2db-707efa5d2b03 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:13.393892 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-97a56471-f54f-4527-b2db-707efa5d2b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:13.397851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-dd4657d9-de1c-4a26-8378-6920d9a96a20 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:13.399004 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-dd4657d9-de1c-4a26-8378-6920d9a96a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:13.403146 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 774/1557] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:13.407945 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 784/1558] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:13.408794 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4657d9-de1c-4a26-8378-6920d9a96a20 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:13.410803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac0571-5ecb-4d56-8ce0-6d7d34a03f1e None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:13.411168 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac0571-5ecb-4d56-8ce0-6d7d34a03f1e None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:13.411810 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac0571-5ecb-4d56-8ce0-6d7d34a03f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:13.420925 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4657d9-de1c-4a26-8378-6920d9a96a20 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:13.424386 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:13.426068 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5099ba9-f5d3-4b58-a8e3-fdca107387d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:13.427012 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5099ba9-f5d3-4b58-a8e3-fdca107387d1 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:13.427261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5099ba9-f5d3-4b58-a8e3-fdca107387d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:13.427531 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5099ba9-f5d3-4b58-a8e3-fdca107387d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:13.428247 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5099ba9-f5d3-4b58-a8e3-fdca107387d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:45:13.428553 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5099ba9-f5d3-4b58-a8e3-fdca107387d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:13.430222 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5099ba9-f5d3-4b58-a8e3-fdca107387d1 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:13.430593 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-b0ac0571-5ecb-4d56-8ce0-6d7d34a03f1e None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:13.432669 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-b0ac0571-5ecb-4d56-8ce0-6d7d34a03f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:13.440776 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 775/1559] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:13.441613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac0571-5ecb-4d56-8ce0-6d7d34a03f1e None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:13.442208 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 785/1560] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:45:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:45:13.443296 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09294938-7eb6-4e20-993d-687426f66570 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:13.443652 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09294938-7eb6-4e20-993d-687426f66570 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:13.444005 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09294938-7eb6-4e20-993d-687426f66570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:13.452469 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac0571-5ecb-4d56-8ce0-6d7d34a03f1e None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:13.457153 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d77d948-6633-4229-be74-fb588cee54fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:13.457928 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77d948-6633-4229-be74-fb588cee54fb admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:13.458648 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77d948-6633-4229-be74-fb588cee54fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:13.458924 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77d948-6633-4229-be74-fb588cee54fb admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:13.459613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d77d948-6633-4229-be74-fb588cee54fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:13.460632 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: WARNING py.warnings [None req-4d77d948-6633-4229-be74-fb588cee54fb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 13:45:13.460632 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: warnings.warn(msg) Jul 06 13:45:13.460632 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: Jul 06 13:45:13.461882 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d77d948-6633-4229-be74-fb588cee54fb admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:13.464080 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-09294938-7eb6-4e20-993d-687426f66570 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:13.465290 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-09294938-7eb6-4e20-993d-687426f66570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:13.473320 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 786/1561] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:13.474071 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09294938-7eb6-4e20-993d-687426f66570 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:13.483732 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09294938-7eb6-4e20-993d-687426f66570 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:13.487780 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 776/1562] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:45:13 2022] POST /identity/v3/users => generated 286 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:13.488448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6651c2-35ac-4de2-9997-4fcfd6e52912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:13.488783 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d77d948-6633-4229-be74-fb588cee54fb admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:13.489102 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6651c2-35ac-4de2-9997-4fcfd6e52912 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:13.489387 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6651c2-35ac-4de2-9997-4fcfd6e52912 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:13.489661 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6651c2-35ac-4de2-9997-4fcfd6e52912 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:13.490375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6651c2-35ac-4de2-9997-4fcfd6e52912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:13.492198 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6651c2-35ac-4de2-9997-4fcfd6e52912 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:13.500140 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d77d948-6633-4229-be74-fb588cee54fb admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:13.505765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 787/1563] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:13.505765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc1834-d3b8-42c4-aeea-08b7211c1a19 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:13.505765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc1834-d3b8-42c4-aeea-08b7211c1a19 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:13.505765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc1834-d3b8-42c4-aeea-08b7211c1a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:13.506153 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65284e7d-671b-47b2-89b6-85f18b68e310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:13.506153 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65284e7d-671b-47b2-89b6-85f18b68e310 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:13.506153 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65284e7d-671b-47b2-89b6-85f18b68e310 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:13.506437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65284e7d-671b-47b2-89b6-85f18b68e310 admin admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:13.507070 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65284e7d-671b-47b2-89b6-85f18b68e310 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:45:13.507407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65284e7d-671b-47b2-89b6-85f18b68e310 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:13.509732 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65284e7d-671b-47b2-89b6-85f18b68e310 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:13.518188 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:13.522272 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 777/1564] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:45:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:45:13.530630 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65284e7d-671b-47b2-89b6-85f18b68e310 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:13.530760 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-30cc1834-d3b8-42c4-aeea-08b7211c1a19 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:13.530760 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-30cc1834-d3b8-42c4-aeea-08b7211c1a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:13.533235 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65284e7d-671b-47b2-89b6-85f18b68e310 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:13.535072 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 788/1565] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:13.535739 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc1834-d3b8-42c4-aeea-08b7211c1a19 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:13.540139 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea56e1f-45d3-4fff-af41-407244cb7d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:13.541447 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea56e1f-45d3-4fff-af41-407244cb7d47 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:13.541866 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea56e1f-45d3-4fff-af41-407244cb7d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:13.542115 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea56e1f-45d3-4fff-af41-407244cb7d47 admin admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:13.543026 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea56e1f-45d3-4fff-af41-407244cb7d47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:45:13.543373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea56e1f-45d3-4fff-af41-407244cb7d47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:13.545230 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea56e1f-45d3-4fff-af41-407244cb7d47 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:13.546230 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc1834-d3b8-42c4-aeea-08b7211c1a19 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:13.550909 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38f157-7d3f-4605-9a43-84673a2c0bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:13.551813 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38f157-7d3f-4605-9a43-84673a2c0bd1 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:13.552099 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38f157-7d3f-4605-9a43-84673a2c0bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:13.552378 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38f157-7d3f-4605-9a43-84673a2c0bd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:13.553159 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38f157-7d3f-4605-9a43-84673a2c0bd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:45:13.553469 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38f157-7d3f-4605-9a43-84673a2c0bd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:13.555035 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38f157-7d3f-4605-9a43-84673a2c0bd1 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:13.558333 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 778/1566] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:45:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:45:13.559364 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea56e1f-45d3-4fff-af41-407244cb7d47 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:13.568739 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 789/1567] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:45:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:45:13.569465 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38f157-7d3f-4605-9a43-84673a2c0bd1 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:13.570134 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea56e1f-45d3-4fff-af41-407244cb7d47 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:13.576010 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc247bf-62a4-4019-88f4-7085179cab18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:13.576768 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc247bf-62a4-4019-88f4-7085179cab18 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:13.577078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc247bf-62a4-4019-88f4-7085179cab18 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:13.577378 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc247bf-62a4-4019-88f4-7085179cab18 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:13.578067 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc247bf-62a4-4019-88f4-7085179cab18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:13.579859 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38f157-7d3f-4605-9a43-84673a2c0bd1 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:13.580003 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc247bf-62a4-4019-88f4-7085179cab18 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:13.584350 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c250901f-93be-4cab-92df-281d2da0f2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:13.584984 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250901f-93be-4cab-92df-281d2da0f2f4 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:13.585301 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250901f-93be-4cab-92df-281d2da0f2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:13.585575 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250901f-93be-4cab-92df-281d2da0f2f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:13.586291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c250901f-93be-4cab-92df-281d2da0f2f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:45:13.586596 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c250901f-93be-4cab-92df-281d2da0f2f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:13.588374 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c250901f-93be-4cab-92df-281d2da0f2f4 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:13.600384 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 790/1568] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:45:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:45:13.608224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c250901f-93be-4cab-92df-281d2da0f2f4 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:13.610851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c250901f-93be-4cab-92df-281d2da0f2f4 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:13.617721 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 779/1569] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:45:13 2022] POST /identity/v3/projects => generated 391 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:13.617876 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f9be42-5f50-43c1-a076-c2ea53d26127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:13.617991 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc247bf-62a4-4019-88f4-7085179cab18 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:13.618096 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9be42-5f50-43c1-a076-c2ea53d26127 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:13.618096 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9be42-5f50-43c1-a076-c2ea53d26127 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:13.618096 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9be42-5f50-43c1-a076-c2ea53d26127 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:13.618096 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f9be42-5f50-43c1-a076-c2ea53d26127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:13.619581 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f9be42-5f50-43c1-a076-c2ea53d26127 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:13.632511 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 791/1570] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:13.632511 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f9be42-5f50-43c1-a076-c2ea53d26127 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:13.640748 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f9be42-5f50-43c1-a076-c2ea53d26127 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:13.681453 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc247bf-62a4-4019-88f4-7085179cab18 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:13.686132 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4152b-196c-4408-a19b-8beae372ea2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:13.687025 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4152b-196c-4408-a19b-8beae372ea2b admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:13.687309 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4152b-196c-4408-a19b-8beae372ea2b admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:13.687581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4152b-196c-4408-a19b-8beae372ea2b admin admin] PATH_INFO: `/v3/projects/d48e24dbeaa241d7a97350fb2c8187b9/users/ad0781ce20e54c3a9b7c61e0cccf6918/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:13.699242 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6505c54b-d4ab-4dd0-aacc-958f26092b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:13.699900 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505c54b-d4ab-4dd0-aacc-958f26092b10 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:13.700180 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505c54b-d4ab-4dd0-aacc-958f26092b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:13.700456 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505c54b-d4ab-4dd0-aacc-958f26092b10 admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:13.701194 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6505c54b-d4ab-4dd0-aacc-958f26092b10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:13.702210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: WARNING py.warnings [None req-6505c54b-d4ab-4dd0-aacc-958f26092b10 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 13:45:13.702210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: warnings.warn(msg) Jul 06 13:45:13.702210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: Jul 06 13:45:13.702642 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4152b-196c-4408-a19b-8beae372ea2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d48e24dbeaa241d7a97350fb2c8187b9, user_id=ad0781ce20e54c3a9b7c61e0cccf6918, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:13.703386 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6505c54b-d4ab-4dd0-aacc-958f26092b10 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:13.705395 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4152b-196c-4408-a19b-8beae372ea2b admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:13.720137 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 780/1571] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:45:13 2022] PUT /identity/v3/projects/d48e24dbeaa241d7a97350fb2c8187b9/users/ad0781ce20e54c3a9b7c61e0cccf6918/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:13.721126 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4152b-196c-4408-a19b-8beae372ea2b admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:13.734335 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 792/1572] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:45:13 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:13.735293 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6505c54b-d4ab-4dd0-aacc-958f26092b10 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:13.786846 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4152b-196c-4408-a19b-8beae372ea2b admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:13.796939 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6505c54b-d4ab-4dd0-aacc-958f26092b10 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:13.801903 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3aa0cb-a1b3-415c-8957-540b1c99ab65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:13.802560 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3aa0cb-a1b3-415c-8957-540b1c99ab65 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:13.802849 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3aa0cb-a1b3-415c-8957-540b1c99ab65 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:13.803133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3aa0cb-a1b3-415c-8957-540b1c99ab65 admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:13.803819 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3aa0cb-a1b3-415c-8957-540b1c99ab65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:13.805732 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3aa0cb-a1b3-415c-8957-540b1c99ab65 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:13.806608 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:13.830508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 793/1573] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:45:13 2022] POST /identity/v3/projects => generated 367 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:13.830508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3aa0cb-a1b3-415c-8957-540b1c99ab65 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:13.845960 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c01d75-24bc-4e1d-80b6-dc0d5772cd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:13.846623 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c01d75-24bc-4e1d-80b6-dc0d5772cd3e admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:13.846903 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c01d75-24bc-4e1d-80b6-dc0d5772cd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:13.847176 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c01d75-24bc-4e1d-80b6-dc0d5772cd3e admin admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:13.847897 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c01d75-24bc-4e1d-80b6-dc0d5772cd3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:45:13.848202 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c01d75-24bc-4e1d-80b6-dc0d5772cd3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:13.849826 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c01d75-24bc-4e1d-80b6-dc0d5772cd3e admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:13.861768 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 781/1574] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:45:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:45:13.865892 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c01d75-24bc-4e1d-80b6-dc0d5772cd3e admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:13.870157 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:13.877573 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c01d75-24bc-4e1d-80b6-dc0d5772cd3e admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:13.900365 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3aa0cb-a1b3-415c-8957-540b1c99ab65 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:13.904990 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f41dce-4892-49c1-a74f-3c26b9cc73a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:13.905695 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f41dce-4892-49c1-a74f-3c26b9cc73a8 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:13.905980 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f41dce-4892-49c1-a74f-3c26b9cc73a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:13.906263 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f41dce-4892-49c1-a74f-3c26b9cc73a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:13.906954 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f41dce-4892-49c1-a74f-3c26b9cc73a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:13.908870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f41dce-4892-49c1-a74f-3c26b9cc73a8 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:13.937074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 794/1575] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:45:13 2022] POST /identity/v3/projects => generated 371 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:13.937302 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893521f-dcb1-493b-b87f-c05f155a6490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:13.937825 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893521f-dcb1-493b-b87f-c05f155a6490 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:13.938007 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f41dce-4892-49c1-a74f-3c26b9cc73a8 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:13.938113 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893521f-dcb1-493b-b87f-c05f155a6490 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:13.938363 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893521f-dcb1-493b-b87f-c05f155a6490 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:13.939051 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8893521f-dcb1-493b-b87f-c05f155a6490 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:13.941273 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8893521f-dcb1-493b-b87f-c05f155a6490 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:13.971298 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 782/1576] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:45:13 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:13.971298 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893521f-dcb1-493b-b87f-c05f155a6490 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:14.003220 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f41dce-4892-49c1-a74f-3c26b9cc73a8 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:14.033736 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893521f-dcb1-493b-b87f-c05f155a6490 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:14.038467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f127a2-398c-4e22-8ace-35e179535e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:14.039134 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f127a2-398c-4e22-8ace-35e179535e49 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:14.039413 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f127a2-398c-4e22-8ace-35e179535e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:14.039683 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f127a2-398c-4e22-8ace-35e179535e49 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:14.040908 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f127a2-398c-4e22-8ace-35e179535e49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:14.042547 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f127a2-398c-4e22-8ace-35e179535e49 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:14.052871 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 783/1577] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:14.058187 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a35da-4adf-47cb-93e7-ca5134753248 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:14.058479 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a35da-4adf-47cb-93e7-ca5134753248 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:14.058758 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a35da-4adf-47cb-93e7-ca5134753248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:14.060525 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd62b72e-2318-4b2f-9edf-6466a622b295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:14.061291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62b72e-2318-4b2f-9edf-6466a622b295 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:14.061571 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62b72e-2318-4b2f-9edf-6466a622b295 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:14.061864 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62b72e-2318-4b2f-9edf-6466a622b295 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:14.062561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd62b72e-2318-4b2f-9edf-6466a622b295 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:14.064453 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd62b72e-2318-4b2f-9edf-6466a622b295 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:14.078340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-995a35da-4adf-47cb-93e7-ca5134753248 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:14.079622 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-995a35da-4adf-47cb-93e7-ca5134753248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:14.086993 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 795/1578] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:45:13 2022] POST /identity/v3/users => generated 296 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:14.086993 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd62b72e-2318-4b2f-9edf-6466a622b295 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:14.089212 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 784/1579] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:14.090050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a35da-4adf-47cb-93e7-ca5134753248 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:14.096426 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd62b72e-2318-4b2f-9edf-6466a622b295 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:14.099844 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a35da-4adf-47cb-93e7-ca5134753248 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:14.101259 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fb140-9afe-4c7e-97e9-cbacb585804e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:14.101915 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fb140-9afe-4c7e-97e9-cbacb585804e admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:14.102198 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fb140-9afe-4c7e-97e9-cbacb585804e admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:14.102475 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fb140-9afe-4c7e-97e9-cbacb585804e admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:14.103705 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78fb140-9afe-4c7e-97e9-cbacb585804e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:14.105245 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616918fa-d18b-4acd-938d-3f9b3c784d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:14.105616 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78fb140-9afe-4c7e-97e9-cbacb585804e admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:14.105911 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616918fa-d18b-4acd-938d-3f9b3c784d5e admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:14.106187 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616918fa-d18b-4acd-938d-3f9b3c784d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:14.106463 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616918fa-d18b-4acd-938d-3f9b3c784d5e admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:14.107143 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616918fa-d18b-4acd-938d-3f9b3c784d5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:14.109112 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616918fa-d18b-4acd-938d-3f9b3c784d5e admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:14.128840 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 796/1580] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:45:14 2022] POST /identity/v3/users => generated 289 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:14.129679 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fb140-9afe-4c7e-97e9-cbacb585804e admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:14.136936 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 785/1581] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:45:14 2022] POST /identity/v3/users => generated 284 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:14.136936 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616918fa-d18b-4acd-938d-3f9b3c784d5e admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:14.139564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fb140-9afe-4c7e-97e9-cbacb585804e admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:14.144208 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369cfee-0057-4ca1-bd72-46da997632be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:14.144869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369cfee-0057-4ca1-bd72-46da997632be admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:14.145212 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369cfee-0057-4ca1-bd72-46da997632be admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:14.145495 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369cfee-0057-4ca1-bd72-46da997632be admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:14.146188 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369cfee-0057-4ca1-bd72-46da997632be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:14.146789 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616918fa-d18b-4acd-938d-3f9b3c784d5e admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:14.148165 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369cfee-0057-4ca1-bd72-46da997632be admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:14.152027 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2cd3de-5715-4259-86ff-3d7a0636cec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:14.152673 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2cd3de-5715-4259-86ff-3d7a0636cec3 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:14.152954 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2cd3de-5715-4259-86ff-3d7a0636cec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:14.153260 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2cd3de-5715-4259-86ff-3d7a0636cec3 admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:14.153934 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2cd3de-5715-4259-86ff-3d7a0636cec3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:14.155815 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2cd3de-5715-4259-86ff-3d7a0636cec3 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:14.171410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:14.181727 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 797/1582] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:45:14 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:14.181727 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369cfee-0057-4ca1-bd72-46da997632be admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:14.182420 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 786/1583] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:45:14 2022] POST /identity/v3/users => generated 286 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:14.183251 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2cd3de-5715-4259-86ff-3d7a0636cec3 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:14.192622 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2cd3de-5715-4259-86ff-3d7a0636cec3 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:14.197914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:14.244550 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369cfee-0057-4ca1-bd72-46da997632be admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:14.249904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594c320-6a83-40a3-ab41-4925e982179e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:14.250241 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb34bf2-6b9c-449d-841d-3639f8b9220b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:14.250795 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594c320-6a83-40a3-ab41-4925e982179e admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:14.250899 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb34bf2-6b9c-449d-841d-3639f8b9220b admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:14.251081 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594c320-6a83-40a3-ab41-4925e982179e admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:14.251188 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb34bf2-6b9c-449d-841d-3639f8b9220b admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:14.251356 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594c320-6a83-40a3-ab41-4925e982179e admin admin] PATH_INFO: `/v3/projects/c4e36049a0fd4b12a1ab55b5b4d5adfe/users/3b07c60a91d241c6b1a42bc32328c274/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:14.251452 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb34bf2-6b9c-449d-841d-3639f8b9220b admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:14.252096 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb34bf2-6b9c-449d-841d-3639f8b9220b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:14.254091 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb34bf2-6b9c-449d-841d-3639f8b9220b admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:14.267503 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f594c320-6a83-40a3-ab41-4925e982179e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4e36049a0fd4b12a1ab55b5b4d5adfe, user_id=3b07c60a91d241c6b1a42bc32328c274, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:14.270201 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f594c320-6a83-40a3-ab41-4925e982179e admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:14.275425 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 798/1584] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:45:14 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:14.276345 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb34bf2-6b9c-449d-841d-3639f8b9220b admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:14.284131 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 787/1585] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:45:14 2022] PUT /identity/v3/projects/c4e36049a0fd4b12a1ab55b5b4d5adfe/users/3b07c60a91d241c6b1a42bc32328c274/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:14.284927 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594c320-6a83-40a3-ab41-4925e982179e admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:14.285814 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb34bf2-6b9c-449d-841d-3639f8b9220b admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:14.356002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba53bf8-dfed-40f0-b46c-5595221ebadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:14.356776 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba53bf8-dfed-40f0-b46c-5595221ebadf admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:14.357093 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba53bf8-dfed-40f0-b46c-5595221ebadf admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:14.357378 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba53bf8-dfed-40f0-b46c-5595221ebadf admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:14.358104 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba53bf8-dfed-40f0-b46c-5595221ebadf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:14.359692 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba53bf8-dfed-40f0-b46c-5595221ebadf admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:14.361812 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594c320-6a83-40a3-ab41-4925e982179e admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:14.366157 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9187c694-f09c-4f5b-bbc1-276ab0887622 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:14.366965 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187c694-f09c-4f5b-bbc1-276ab0887622 service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:14.367250 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187c694-f09c-4f5b-bbc1-276ab0887622 service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:14.367679 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187c694-f09c-4f5b-bbc1-276ab0887622 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:14.373059 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 799/1586] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:14.373059 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba53bf8-dfed-40f0-b46c-5595221ebadf admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:14.380506 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9187c694-f09c-4f5b-bbc1-276ab0887622 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:14.381974 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9187c694-f09c-4f5b-bbc1-276ab0887622 service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:14.382116 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba53bf8-dfed-40f0-b46c-5595221ebadf admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:14.387259 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c909bea-c67d-4fba-94cf-646a3a2b6e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:14.388043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c909bea-c67d-4fba-94cf-646a3a2b6e03 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:14.388333 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c909bea-c67d-4fba-94cf-646a3a2b6e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:14.388608 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c909bea-c67d-4fba-94cf-646a3a2b6e03 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:14.389347 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c909bea-c67d-4fba-94cf-646a3a2b6e03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:14.390903 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c909bea-c67d-4fba-94cf-646a3a2b6e03 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:14.392451 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 788/1587] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:14.393348 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9187c694-f09c-4f5b-bbc1-276ab0887622 service neutron] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:14.401880 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 800/1588] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:14.402759 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c909bea-c67d-4fba-94cf-646a3a2b6e03 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:14.405986 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9187c694-f09c-4f5b-bbc1-276ab0887622 service neutron] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:14.410405 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b1d0c-9de1-477f-87ad-fda8a5db9d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:14.411070 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b1d0c-9de1-477f-87ad-fda8a5db9d61 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:14.411351 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b1d0c-9de1-477f-87ad-fda8a5db9d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:14.411628 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b1d0c-9de1-477f-87ad-fda8a5db9d61 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:14.412344 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026b1d0c-9de1-477f-87ad-fda8a5db9d61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:14.412667 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c909bea-c67d-4fba-94cf-646a3a2b6e03 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:14.414132 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026b1d0c-9de1-477f-87ad-fda8a5db9d61 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:14.421186 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f24411-d82e-4613-9ddd-0c4b3b9a8c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:14.422033 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f24411-d82e-4613-9ddd-0c4b3b9a8c98 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:14.422176 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f24411-d82e-4613-9ddd-0c4b3b9a8c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:14.422576 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f24411-d82e-4613-9ddd-0c4b3b9a8c98 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:14.423200 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f24411-d82e-4613-9ddd-0c4b3b9a8c98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:14.425160 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f24411-d82e-4613-9ddd-0c4b3b9a8c98 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:14.428938 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 789/1589] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:14.428938 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b1d0c-9de1-477f-87ad-fda8a5db9d61 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:14.439589 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b1d0c-9de1-477f-87ad-fda8a5db9d61 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:14.444851 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68282e-9dcf-4196-9eb8-058e7158971e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:14.445593 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68282e-9dcf-4196-9eb8-058e7158971e admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:14.446079 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68282e-9dcf-4196-9eb8-058e7158971e admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:14.446538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68282e-9dcf-4196-9eb8-058e7158971e admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:14.447778 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a68282e-9dcf-4196-9eb8-058e7158971e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:14.449645 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a68282e-9dcf-4196-9eb8-058e7158971e admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:14.457091 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 801/1590] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:45:14 2022] POST /identity/v3/users => generated 293 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:14.458049 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f24411-d82e-4613-9ddd-0c4b3b9a8c98 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:14.467030 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 790/1591] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:14.467030 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55cdbd-d03c-4f56-8c0b-6ab563c1ca38 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:14.467030 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55cdbd-d03c-4f56-8c0b-6ab563c1ca38 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:14.467030 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55cdbd-d03c-4f56-8c0b-6ab563c1ca38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:14.472204 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f24411-d82e-4613-9ddd-0c4b3b9a8c98 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:14.477881 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69aa136-b97f-4396-a033-5dfeb7b55775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:14.478829 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69aa136-b97f-4396-a033-5dfeb7b55775 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:14.479644 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69aa136-b97f-4396-a033-5dfeb7b55775 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:14.479644 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69aa136-b97f-4396-a033-5dfeb7b55775 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:14.480398 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69aa136-b97f-4396-a033-5dfeb7b55775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:14.482351 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69aa136-b97f-4396-a033-5dfeb7b55775 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:14.498020 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 802/1592] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:14.498020 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69aa136-b97f-4396-a033-5dfeb7b55775 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:14.500511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-7f55cdbd-d03c-4f56-8c0b-6ab563c1ca38 None None] MFA Rules not processed for user `3b07c60a91d241c6b1a42bc32328c274`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:14.502031 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-7f55cdbd-d03c-4f56-8c0b-6ab563c1ca38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:14.506288 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69aa136-b97f-4396-a033-5dfeb7b55775 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:14.511594 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366211f-5d5a-4a53-b5e4-fdf0a3f2d0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:14.512329 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366211f-5d5a-4a53-b5e4-fdf0a3f2d0fd admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:14.512721 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366211f-5d5a-4a53-b5e4-fdf0a3f2d0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:14.513207 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366211f-5d5a-4a53-b5e4-fdf0a3f2d0fd admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:14.514003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5366211f-5d5a-4a53-b5e4-fdf0a3f2d0fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:14.515590 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5366211f-5d5a-4a53-b5e4-fdf0a3f2d0fd admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:14.529113 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 803/1593] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:14.533273 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366211f-5d5a-4a53-b5e4-fdf0a3f2d0fd admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:14.543556 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366211f-5d5a-4a53-b5e4-fdf0a3f2d0fd admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:14.548595 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd411b5-674e-4952-8431-b7c456710f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:14.549291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd411b5-674e-4952-8431-b7c456710f23 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:14.549616 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd411b5-674e-4952-8431-b7c456710f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:14.549835 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd411b5-674e-4952-8431-b7c456710f23 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:14.550560 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd411b5-674e-4952-8431-b7c456710f23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:14.552401 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd411b5-674e-4952-8431-b7c456710f23 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:14.562334 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:14.565427 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 804/1594] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:14.566320 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd411b5-674e-4952-8431-b7c456710f23 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:14.578585 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd411b5-674e-4952-8431-b7c456710f23 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:14.584838 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aeda12-6302-4dde-9488-82fd43308562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:14.585816 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aeda12-6302-4dde-9488-82fd43308562 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:14.586177 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aeda12-6302-4dde-9488-82fd43308562 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:14.586582 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aeda12-6302-4dde-9488-82fd43308562 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:14.587510 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aeda12-6302-4dde-9488-82fd43308562 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:14.589458 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aeda12-6302-4dde-9488-82fd43308562 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:14.592902 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 791/1595] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:14.593998 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55cdbd-d03c-4f56-8c0b-6ab563c1ca38 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:14.605753 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 805/1596] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:14.606684 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aeda12-6302-4dde-9488-82fd43308562 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:14.626958 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aeda12-6302-4dde-9488-82fd43308562 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:14.631123 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55cdbd-d03c-4f56-8c0b-6ab563c1ca38 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:14.638534 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c6801-e610-4cac-8900-4a23460e00f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:14.639357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c6801-e610-4cac-8900-4a23460e00f5 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:14.639848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c6801-e610-4cac-8900-4a23460e00f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:14.640189 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c6801-e610-4cac-8900-4a23460e00f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:14.641121 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c6801-e610-4cac-8900-4a23460e00f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:14.643714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d8bf08-cc0c-41eb-b264-9750bd74066f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:14.644339 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c6801-e610-4cac-8900-4a23460e00f5 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:14.646121 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d8bf08-cc0c-41eb-b264-9750bd74066f admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:14.646478 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d8bf08-cc0c-41eb-b264-9750bd74066f admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:14.647675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d8bf08-cc0c-41eb-b264-9750bd74066f admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:14.648589 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d8bf08-cc0c-41eb-b264-9750bd74066f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:14.650533 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d8bf08-cc0c-41eb-b264-9750bd74066f admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:14.660630 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 792/1597] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:14.660630 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c6801-e610-4cac-8900-4a23460e00f5 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:14.664709 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 806/1598] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:14.666664 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d8bf08-cc0c-41eb-b264-9750bd74066f admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:14.671326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c6801-e610-4cac-8900-4a23460e00f5 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:14.677689 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d8bf08-cc0c-41eb-b264-9750bd74066f admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:14.678494 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2fb94-1bef-47e4-b8b9-14b11e06bf3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:14.679380 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2fb94-1bef-47e4-b8b9-14b11e06bf3f admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:14.679667 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2fb94-1bef-47e4-b8b9-14b11e06bf3f admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:14.679953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2fb94-1bef-47e4-b8b9-14b11e06bf3f admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:14.680663 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2fb94-1bef-47e4-b8b9-14b11e06bf3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:14.683099 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2fb94-1bef-47e4-b8b9-14b11e06bf3f admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:14.684893 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a9fb9-6a07-4c2b-9e63-f6fb748b1538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:14.686383 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a9fb9-6a07-4c2b-9e63-f6fb748b1538 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:14.686957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a9fb9-6a07-4c2b-9e63-f6fb748b1538 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:14.687417 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a9fb9-6a07-4c2b-9e63-f6fb748b1538 admin admin] PATH_INFO: `/v3/projects/3b2a80a004e04da7ae12db9f3f87146f/users/0c83c05bec4f456d8aa7d51732ddc8af/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:14.693963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 793/1599] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:14.694853 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2fb94-1bef-47e4-b8b9-14b11e06bf3f admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:14.709264 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2fb94-1bef-47e4-b8b9-14b11e06bf3f admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:14.714091 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a9fb9-6a07-4c2b-9e63-f6fb748b1538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b2a80a004e04da7ae12db9f3f87146f, user_id=0c83c05bec4f456d8aa7d51732ddc8af, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:14.714793 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: WARNING py.warnings [None req-b78a9fb9-6a07-4c2b-9e63-f6fb748b1538 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 13:45:14.714793 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: warnings.warn(msg) Jul 06 13:45:14.714793 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: Jul 06 13:45:14.716101 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58cef65-edfe-4362-9bd4-c4065fc81716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:14.716787 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a9fb9-6a07-4c2b-9e63-f6fb748b1538 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:14.717029 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58cef65-edfe-4362-9bd4-c4065fc81716 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:14.717319 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58cef65-edfe-4362-9bd4-c4065fc81716 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:14.717606 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58cef65-edfe-4362-9bd4-c4065fc81716 admin admin] PATH_INFO: `/v3/projects/05df572327d54bd99595974b75898052/users/3d7cacb73c8342579d9f990b0193487c/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:14.719218 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:14.731647 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 807/1600] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:45:14 2022] PUT /identity/v3/projects/3b2a80a004e04da7ae12db9f3f87146f/users/0c83c05bec4f456d8aa7d51732ddc8af/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:14.732525 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a9fb9-6a07-4c2b-9e63-f6fb748b1538 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:14.734330 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58cef65-edfe-4362-9bd4-c4065fc81716 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05df572327d54bd99595974b75898052, user_id=3d7cacb73c8342579d9f990b0193487c, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:14.737388 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58cef65-edfe-4362-9bd4-c4065fc81716 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:14.759459 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 794/1601] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:45:14 2022] PUT /identity/v3/projects/05df572327d54bd99595974b75898052/users/3d7cacb73c8342579d9f990b0193487c/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:14.767965 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f991f-6fb3-40ee-8c80-33207fc0059a None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:14.769269 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f991f-6fb3-40ee-8c80-33207fc0059a None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:14.769608 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f991f-6fb3-40ee-8c80-33207fc0059a None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:14.773421 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 795/1602] 10.222.0.20 () {54 vars in 961 bytes} [Wed Jul 6 13:45:14 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:45:14.774307 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f991f-6fb3-40ee-8c80-33207fc0059a None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:14.819305 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a9fb9-6a07-4c2b-9e63-f6fb748b1538 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:14.863539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f991f-6fb3-40ee-8c80-33207fc0059a None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:14.871015 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64dd412-9ae2-4aa3-b08b-71e719ddeb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:14.871816 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64dd412-9ae2-4aa3-b08b-71e719ddeb14 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:14.872111 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64dd412-9ae2-4aa3-b08b-71e719ddeb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:14.872551 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64dd412-9ae2-4aa3-b08b-71e719ddeb14 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:14.873365 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64dd412-9ae2-4aa3-b08b-71e719ddeb14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:14.874994 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64dd412-9ae2-4aa3-b08b-71e719ddeb14 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:14.883555 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab0f39-a8e3-4d06-a642-eaa303abcc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:14.885164 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab0f39-a8e3-4d06-a642-eaa303abcc4f admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:14.885777 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab0f39-a8e3-4d06-a642-eaa303abcc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:14.886073 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab0f39-a8e3-4d06-a642-eaa303abcc4f admin admin] PATH_INFO: `/v3/projects/7c55552b8bb945818f81c7ed751bf6cf/users/0147bf710e1a42b79668bbd6936c4e60/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:14.887204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 796/1603] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:14.888019 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64dd412-9ae2-4aa3-b08b-71e719ddeb14 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:14.898093 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64dd412-9ae2-4aa3-b08b-71e719ddeb14 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:14.904756 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd234b-5c4c-447b-8098-3b2d58ecebb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:14.905786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd234b-5c4c-447b-8098-3b2d58ecebb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:14.906075 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd234b-5c4c-447b-8098-3b2d58ecebb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:14.906354 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd234b-5c4c-447b-8098-3b2d58ecebb9 admin admin] PATH_INFO: `/v3/projects/ba428e3a1e094f7c83fae3f3d5f6e8c2/users/9d13fb415ee54beea237ed2258c89691/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:14.907449 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ab0f39-a8e3-4d06-a642-eaa303abcc4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c55552b8bb945818f81c7ed751bf6cf, user_id=0147bf710e1a42b79668bbd6936c4e60, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:14.910363 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ab0f39-a8e3-4d06-a642-eaa303abcc4f admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:14.923286 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd234b-5c4c-447b-8098-3b2d58ecebb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba428e3a1e094f7c83fae3f3d5f6e8c2, user_id=9d13fb415ee54beea237ed2258c89691, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:14.926469 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd234b-5c4c-447b-8098-3b2d58ecebb9 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:14.937963 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 808/1604] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:45:14 2022] PUT /identity/v3/projects/7c55552b8bb945818f81c7ed751bf6cf/users/0147bf710e1a42b79668bbd6936c4e60/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:14.937963 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab0f39-a8e3-4d06-a642-eaa303abcc4f admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:14.961873 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 797/1605] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:45:14 2022] PUT /identity/v3/projects/ba428e3a1e094f7c83fae3f3d5f6e8c2/users/9d13fb415ee54beea237ed2258c89691/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:14.961873 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d955b-a5ab-4b62-824f-7798a809f5f6 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:14.961873 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d955b-a5ab-4b62-824f-7798a809f5f6 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:14.961873 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d955b-a5ab-4b62-824f-7798a809f5f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:14.976048 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-547d955b-a5ab-4b62-824f-7798a809f5f6 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:14.977366 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-547d955b-a5ab-4b62-824f-7798a809f5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:15.017454 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab0f39-a8e3-4d06-a642-eaa303abcc4f admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:15.044753 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 798/1606] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:15.047424 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb6ddb-526d-468a-9fa7-fe3a1bb52da3 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:15.047885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb6ddb-526d-468a-9fa7-fe3a1bb52da3 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:15.048270 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb6ddb-526d-468a-9fa7-fe3a1bb52da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:15.058966 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:15.068064 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-a6eb6ddb-526d-468a-9fa7-fe3a1bb52da3 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:15.069340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a6eb6ddb-526d-468a-9fa7-fe3a1bb52da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:15.078253 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 799/1607] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:15.083042 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aec0ad-e57d-40f3-84f9-d7468bfa0833 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:15.083042 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aec0ad-e57d-40f3-84f9-d7468bfa0833 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:15.083042 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aec0ad-e57d-40f3-84f9-d7468bfa0833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:15.100136 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fede159-000f-4c6d-8177-f224821fdeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:15.101085 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fede159-000f-4c6d-8177-f224821fdeb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:15.101422 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fede159-000f-4c6d-8177-f224821fdeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:15.101679 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fede159-000f-4c6d-8177-f224821fdeb6 admin admin] PATH_INFO: `/v3/projects/5080a0e6cfb24d799c3b31cec1d073bc/users/ae1587d0dc6246f9997824193eef6d78/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:15.102834 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-f5aec0ad-e57d-40f3-84f9-d7468bfa0833 None None] MFA Rules not processed for user `6647eb1e8a1544ef9d3a9806f2d1fc51`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:15.104234 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-f5aec0ad-e57d-40f3-84f9-d7468bfa0833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:15.110120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:15.128529 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fede159-000f-4c6d-8177-f224821fdeb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5080a0e6cfb24d799c3b31cec1d073bc, user_id=ae1587d0dc6246f9997824193eef6d78, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:15.132241 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fede159-000f-4c6d-8177-f224821fdeb6 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:15.160029 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 809/1608] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:45:14 2022] PUT /identity/v3/projects/5080a0e6cfb24d799c3b31cec1d073bc/users/ae1587d0dc6246f9997824193eef6d78/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:15.161248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fede159-000f-4c6d-8177-f224821fdeb6 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:15.261518 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fede159-000f-4c6d-8177-f224821fdeb6 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:15.270100 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2869ed-557a-4494-9c1a-6b7ad025aae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:15.271025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2869ed-557a-4494-9c1a-6b7ad025aae5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:15.271315 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2869ed-557a-4494-9c1a-6b7ad025aae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:15.271785 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2869ed-557a-4494-9c1a-6b7ad025aae5 admin admin] PATH_INFO: `/v3/projects/bd85f2a57a0847debf5aaf81aab7a638/users/7b572a8f465048558fc969b85b2aec5b/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:15.293728 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2869ed-557a-4494-9c1a-6b7ad025aae5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd85f2a57a0847debf5aaf81aab7a638, user_id=7b572a8f465048558fc969b85b2aec5b, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:15.297316 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2869ed-557a-4494-9c1a-6b7ad025aae5 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:15.318249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 810/1609] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:45:15 2022] PUT /identity/v3/projects/bd85f2a57a0847debf5aaf81aab7a638/users/7b572a8f465048558fc969b85b2aec5b/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:15.320398 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0ff2b-35e3-408a-bcb4-3791ea537380 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:15.320790 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0ff2b-35e3-408a-bcb4-3791ea537380 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:15.321312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0ff2b-35e3-408a-bcb4-3791ea537380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:15.353832 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-20c0ff2b-35e3-408a-bcb4-3791ea537380 None None] MFA Rules not processed for user `0147bf710e1a42b79668bbd6936c4e60`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:15.355043 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 800/1610] 10.222.0.20 () {58 vars in 1058 bytes} [Wed Jul 6 13:45:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:15.357315 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b244be-5f0b-493b-93eb-ca4c51b7bf55 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:15.357607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b244be-5f0b-493b-93eb-ca4c51b7bf55 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:15.357887 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b244be-5f0b-493b-93eb-ca4c51b7bf55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:15.362884 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-20c0ff2b-35e3-408a-bcb4-3791ea537380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:15.393312 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-b5b244be-5f0b-493b-93eb-ca4c51b7bf55 None None] MFA Rules not processed for user `9d13fb415ee54beea237ed2258c89691`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:15.395827 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-b5b244be-5f0b-493b-93eb-ca4c51b7bf55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:15.442992 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 811/1611] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:15 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:15.445295 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0ff2b-35e3-408a-bcb4-3791ea537380 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:15.452946 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-20c0ff2b-35e3-408a-bcb4-3791ea537380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:15.491146 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 801/1612] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:15.492378 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b244be-5f0b-493b-93eb-ca4c51b7bf55 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:15.494317 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-b5b244be-5f0b-493b-93eb-ca4c51b7bf55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:15.502124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:15.537913 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0ff2b-35e3-408a-bcb4-3791ea537380 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:15.544263 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eb5d32-cfd9-4fff-bb3c-7fadfe45f35b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:15.545395 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb5d32-cfd9-4fff-bb3c-7fadfe45f35b service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:15.545395 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb5d32-cfd9-4fff-bb3c-7fadfe45f35b service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:15.545664 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb5d32-cfd9-4fff-bb3c-7fadfe45f35b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:15.572661 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b244be-5f0b-493b-93eb-ca4c51b7bf55 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:15.577808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1e3ada-d458-4dfe-a061-408102ffd315 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:15.579005 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e3ada-d458-4dfe-a061-408102ffd315 service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:15.579295 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e3ada-d458-4dfe-a061-408102ffd315 service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:15.579937 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e3ada-d458-4dfe-a061-408102ffd315 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:15.624762 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eb5d32-cfd9-4fff-bb3c-7fadfe45f35b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:15.626793 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eb5d32-cfd9-4fff-bb3c-7fadfe45f35b service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:15.632989 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:15.640133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 812/1613] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:15.641448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa71ef-7876-433c-bd08-e191f606a1f2 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:15.641801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa71ef-7876-433c-bd08-e191f606a1f2 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:15.642151 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa71ef-7876-433c-bd08-e191f606a1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:15.654295 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1e3ada-d458-4dfe-a061-408102ffd315 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:15.655850 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1e3ada-d458-4dfe-a061-408102ffd315 service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:15.668685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 802/1614] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:15.670064 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d98d3-bec5-4385-967b-eff9bea1101b None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:15.670355 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d98d3-bec5-4385-967b-eff9bea1101b None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:15.670631 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d98d3-bec5-4385-967b-eff9bea1101b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:15.680914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-8caa71ef-7876-433c-bd08-e191f606a1f2 None None] MFA Rules not processed for user `ae1587d0dc6246f9997824193eef6d78`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:15.683656 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-8caa71ef-7876-433c-bd08-e191f606a1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:15.697670 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-927d98d3-bec5-4385-967b-eff9bea1101b None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:15.698971 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-927d98d3-bec5-4385-967b-eff9bea1101b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:15.709115 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 803/1615] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:15.709115 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d98d3-bec5-4385-967b-eff9bea1101b None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:15.803210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 813/1616] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:15.804181 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa71ef-7876-433c-bd08-e191f606a1f2 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:15.839576 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d98d3-bec5-4385-967b-eff9bea1101b None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:15.849661 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe786ff-1c63-4ac7-86f8-39d6076defd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:15.850359 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe786ff-1c63-4ac7-86f8-39d6076defd2 service nova] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:15.850643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe786ff-1c63-4ac7-86f8-39d6076defd2 service nova] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:15.850974 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe786ff-1c63-4ac7-86f8-39d6076defd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:15.913070 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa71ef-7876-433c-bd08-e191f606a1f2 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:15.918984 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c741c-da7a-4152-9a74-101ed9fc3807 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:15.919867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c741c-da7a-4152-9a74-101ed9fc3807 service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:15.920333 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c741c-da7a-4152-9a74-101ed9fc3807 service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:15.920775 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c741c-da7a-4152-9a74-101ed9fc3807 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:15.922628 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe786ff-1c63-4ac7-86f8-39d6076defd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:15.924312 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe786ff-1c63-4ac7-86f8-39d6076defd2 service nova] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:15.934575 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c741c-da7a-4152-9a74-101ed9fc3807 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:15.934920 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 804/1617] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:15.935756 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe786ff-1c63-4ac7-86f8-39d6076defd2 service nova] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:15.946115 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe786ff-1c63-4ac7-86f8-39d6076defd2 service nova] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:15.948789 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c741c-da7a-4152-9a74-101ed9fc3807 service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:15.960325 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f26ca-4b8c-4757-9437-c441ef3c8069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:15.961011 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f26ca-4b8c-4757-9437-c441ef3c8069 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:15.961317 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f26ca-4b8c-4757-9437-c441ef3c8069 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:15.961711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f26ca-4b8c-4757-9437-c441ef3c8069 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:15.962968 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f26ca-4b8c-4757-9437-c441ef3c8069 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:15.964867 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f26ca-4b8c-4757-9437-c441ef3c8069 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:15.968515 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 814/1618] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:15 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:15.969659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c741c-da7a-4152-9a74-101ed9fc3807 service cinder] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:15.996938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c741c-da7a-4152-9a74-101ed9fc3807 service cinder] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:16.003450 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c63fc1-680c-4d2a-afdd-bc4e42a32fee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:16.004364 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c63fc1-680c-4d2a-afdd-bc4e42a32fee service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:16.004943 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c63fc1-680c-4d2a-afdd-bc4e42a32fee service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:16.005493 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c63fc1-680c-4d2a-afdd-bc4e42a32fee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:16.016326 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:16.019892 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c63fc1-680c-4d2a-afdd-bc4e42a32fee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:16.024363 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 805/1619] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:45:15 2022] POST /identity/v3/projects => generated 373 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:16.024363 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c1e8a-3ddf-48c9-86d2-f9a3099e2b99 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:16.024363 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c1e8a-3ddf-48c9-86d2-f9a3099e2b99 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:16.024363 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c1e8a-3ddf-48c9-86d2-f9a3099e2b99 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:16.026061 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c63fc1-680c-4d2a-afdd-bc4e42a32fee service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:16.035902 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 806/1620] 10.222.0.20 () {54 vars in 961 bytes} [Wed Jul 6 13:45:16 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:45:16.035902 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1c1e8a-3ddf-48c9-86d2-f9a3099e2b99 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:16.044118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 815/1621] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:16.046184 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4f20c-eca8-4fb4-9674-0e6cb8b5234b None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:16.046681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4f20c-eca8-4fb4-9674-0e6cb8b5234b None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:16.047145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4f20c-eca8-4fb4-9674-0e6cb8b5234b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:16.070455 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-0da4f20c-eca8-4fb4-9674-0e6cb8b5234b None None] MFA Rules not processed for user `6647eb1e8a1544ef9d3a9806f2d1fc51`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:16.074592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-0da4f20c-eca8-4fb4-9674-0e6cb8b5234b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:16.179924 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:16.206065 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1c1e8a-3ddf-48c9-86d2-f9a3099e2b99 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:16.211381 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba1fe4c-9ce2-4c30-ad21-0644b9530167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:16.212103 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1fe4c-9ce2-4c30-ad21-0644b9530167 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:16.213425 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1fe4c-9ce2-4c30-ad21-0644b9530167 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:16.213425 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1fe4c-9ce2-4c30-ad21-0644b9530167 admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:16.214183 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba1fe4c-9ce2-4c30-ad21-0644b9530167 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:16.216221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 816/1622] 10.222.0.20 () {58 vars in 1058 bytes} [Wed Jul 6 13:45:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:16.225921 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba1fe4c-9ce2-4c30-ad21-0644b9530167 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:16.229769 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4f20c-eca8-4fb4-9674-0e6cb8b5234b None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:16.260754 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4f20c-eca8-4fb4-9674-0e6cb8b5234b None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:16.261815 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 807/1623] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:45:16 2022] POST /identity/v3/users => generated 293 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:16.266770 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba1fe4c-9ce2-4c30-ad21-0644b9530167 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:16.280036 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06094fee-7b81-4e49-948d-5f2381ab1f6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:16.283455 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06094fee-7b81-4e49-948d-5f2381ab1f6f service nova] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:16.283824 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06094fee-7b81-4e49-948d-5f2381ab1f6f service nova] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:16.284958 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06094fee-7b81-4e49-948d-5f2381ab1f6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:16.305872 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba1fe4c-9ce2-4c30-ad21-0644b9530167 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:16.319411 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68cd9d-c3e0-41cc-ab54-1fff55105324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:16.320960 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68cd9d-c3e0-41cc-ab54-1fff55105324 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:16.321270 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68cd9d-c3e0-41cc-ab54-1fff55105324 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:16.322245 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68cd9d-c3e0-41cc-ab54-1fff55105324 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:16.327000 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68cd9d-c3e0-41cc-ab54-1fff55105324 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:16.330468 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68cd9d-c3e0-41cc-ab54-1fff55105324 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:16.437439 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 808/1624] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:45:16.437439 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68cd9d-c3e0-41cc-ab54-1fff55105324 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:16.437439 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68cd9d-c3e0-41cc-ab54-1fff55105324 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:16.437439 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7a4a8-7db3-4737-b0f4-4907c974a28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:16.439116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7a4a8-7db3-4737-b0f4-4907c974a28e admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:16.439116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7a4a8-7db3-4737-b0f4-4907c974a28e admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:16.439116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7a4a8-7db3-4737-b0f4-4907c974a28e admin admin] PATH_INFO: `/v3/projects/7f26915cd1f540a6a3ecc3ff1e370f82/users/dd7a4a682d344a6e81cedf1c1d1a19e1/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:16.501668 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e7a4a8-7db3-4737-b0f4-4907c974a28e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f26915cd1f540a6a3ecc3ff1e370f82, user_id=dd7a4a682d344a6e81cedf1c1d1a19e1, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:16.512712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e7a4a8-7db3-4737-b0f4-4907c974a28e admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:16.520511 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06094fee-7b81-4e49-948d-5f2381ab1f6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:16.526695 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06094fee-7b81-4e49-948d-5f2381ab1f6f service nova] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:16.542685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 809/1625] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:45:16 2022] PUT /identity/v3/projects/7f26915cd1f540a6a3ecc3ff1e370f82/users/dd7a4a682d344a6e81cedf1c1d1a19e1/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:16.547565 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a6f5d-8d9f-477c-9a87-cfe37a25c15d None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:16.548235 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a6f5d-8d9f-477c-9a87-cfe37a25c15d None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:16.548521 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a6f5d-8d9f-477c-9a87-cfe37a25c15d None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:16.551876 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 810/1626] 10.222.0.20 () {54 vars in 912 bytes} [Wed Jul 6 13:45:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:45:16.552682 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a6f5d-8d9f-477c-9a87-cfe37a25c15d None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:16.565299 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 817/1627] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:16.567146 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5402b97-2e08-435b-9bd6-0c39bd7cfe6d None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:16.567441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5402b97-2e08-435b-9bd6-0c39bd7cfe6d None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:16.567729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5402b97-2e08-435b-9bd6-0c39bd7cfe6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:16.604970 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-f5402b97-2e08-435b-9bd6-0c39bd7cfe6d None None] MFA Rules not processed for user `6647eb1e8a1544ef9d3a9806f2d1fc51`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:16.606135 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-f5402b97-2e08-435b-9bd6-0c39bd7cfe6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:16.729165 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 818/1628] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Jul 6 13:45:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:45:16.744041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5402b97-2e08-435b-9bd6-0c39bd7cfe6d None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:16.745169 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a6f5d-8d9f-477c-9a87-cfe37a25c15d None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:16.750371 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca70db-9950-4c5f-b584-b56726022e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:16.751234 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca70db-9950-4c5f-b584-b56726022e2e admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:16.752609 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca70db-9950-4c5f-b584-b56726022e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:16.752765 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca70db-9950-4c5f-b584-b56726022e2e admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:16.753564 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca70db-9950-4c5f-b584-b56726022e2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:16.755188 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca70db-9950-4c5f-b584-b56726022e2e admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:16.767236 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 811/1629] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:16.772431 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca70db-9950-4c5f-b584-b56726022e2e admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:16.796235 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca70db-9950-4c5f-b584-b56726022e2e admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:16.803096 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:16.812888 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad536af0-cc81-418a-9bca-042cd186e299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:16.814018 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad536af0-cc81-418a-9bca-042cd186e299 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:16.814494 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad536af0-cc81-418a-9bca-042cd186e299 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:16.814804 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad536af0-cc81-418a-9bca-042cd186e299 admin admin] PATH_INFO: `/v3/projects/7f26915cd1f540a6a3ecc3ff1e370f82/users/dd7a4a682d344a6e81cedf1c1d1a19e1/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:16.819485 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad536af0-cc81-418a-9bca-042cd186e299 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f26915cd1f540a6a3ecc3ff1e370f82, user_id=dd7a4a682d344a6e81cedf1c1d1a19e1, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:16.826353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad536af0-cc81-418a-9bca-042cd186e299 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:16.867116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 812/1630] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:45:16 2022] PUT /identity/v3/projects/7f26915cd1f540a6a3ecc3ff1e370f82/users/dd7a4a682d344a6e81cedf1c1d1a19e1/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:16.869768 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942f116-b8dc-4056-9add-b4db4937021f None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:16.869929 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942f116-b8dc-4056-9add-b4db4937021f None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:16.870213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942f116-b8dc-4056-9add-b4db4937021f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:16.894218 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5402b97-2e08-435b-9bd6-0c39bd7cfe6d None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:16.940063 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-6942f116-b8dc-4056-9add-b4db4937021f None None] MFA Rules not processed for user `dd7a4a682d344a6e81cedf1c1d1a19e1`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:16.942194 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-6942f116-b8dc-4056-9add-b4db4937021f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:16.973837 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f7307-bbcf-4e79-acb0-bf06936d9575 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:16.974609 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f7307-bbcf-4e79-acb0-bf06936d9575 service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:16.975019 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f7307-bbcf-4e79-acb0-bf06936d9575 service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:16.975317 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f7307-bbcf-4e79-acb0-bf06936d9575 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:17.169293 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f7307-bbcf-4e79-acb0-bf06936d9575 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:17.170766 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f7307-bbcf-4e79-acb0-bf06936d9575 service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:17.213822 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 813/1631] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:16 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:45:17.222165 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4bb57e-7f59-48ec-88b4-8383cb60474c None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:17.222361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4bb57e-7f59-48ec-88b4-8383cb60474c None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:17.222648 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4bb57e-7f59-48ec-88b4-8383cb60474c None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:17.224891 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 819/1632] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 13:45:17.226653 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 814/1633] 10.222.0.20 () {54 vars in 912 bytes} [Wed Jul 6 13:45:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:45:17.230281 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d2017-e68d-4c68-8671-4777b7f640e3 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:17.230575 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d2017-e68d-4c68-8671-4777b7f640e3 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:17.230875 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d2017-e68d-4c68-8671-4777b7f640e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:17.242092 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4bb57e-7f59-48ec-88b4-8383cb60474c None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:17.256410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4bb57e-7f59-48ec-88b4-8383cb60474c None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:17.268502 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865bdff-7364-4579-b879-065e9239ac2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:17.269320 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865bdff-7364-4579-b879-065e9239ac2d service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:17.269613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865bdff-7364-4579-b879-065e9239ac2d service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:17.270317 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865bdff-7364-4579-b879-065e9239ac2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:17.282476 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-da8d2017-e68d-4c68-8671-4777b7f640e3 None None] MFA Rules not processed for user `6647eb1e8a1544ef9d3a9806f2d1fc51`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:17.285304 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b865bdff-7364-4579-b879-065e9239ac2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:17.287055 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b865bdff-7364-4579-b879-065e9239ac2d service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:17.288258 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-da8d2017-e68d-4c68-8671-4777b7f640e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:17.303834 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:17.316783 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 815/1634] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:17 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 13:45:17.320084 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 820/1635] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Jul 6 13:45:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:17.380465 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865bdff-7364-4579-b879-065e9239ac2d service cinder] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:17.394465 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865bdff-7364-4579-b879-065e9239ac2d service cinder] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:17.398715 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d40626-201c-48b2-a6d2-9e9e037409a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:17.401284 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d40626-201c-48b2-a6d2-9e9e037409a3 service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:17.401284 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d40626-201c-48b2-a6d2-9e9e037409a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:17.401284 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d40626-201c-48b2-a6d2-9e9e037409a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:17.417463 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d40626-201c-48b2-a6d2-9e9e037409a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:17.419270 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d40626-201c-48b2-a6d2-9e9e037409a3 service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:17.450223 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 816/1636] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:18.187384 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d2017-e68d-4c68-8671-4777b7f640e3 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:18.220904 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d2017-e68d-4c68-8671-4777b7f640e3 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:18.237879 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec74ea-a651-44f3-baea-e491694de07c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:18.238755 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec74ea-a651-44f3-baea-e491694de07c service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:18.239051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec74ea-a651-44f3-baea-e491694de07c service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:18.239332 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec74ea-a651-44f3-baea-e491694de07c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:18.416223 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec74ea-a651-44f3-baea-e491694de07c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:18.418470 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec74ea-a651-44f3-baea-e491694de07c service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:18.455994 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 821/1637] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:24.689201 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b23a61-3a91-4b78-b6e6-3cb989dff4f0 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:24.689717 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b23a61-3a91-4b78-b6e6-3cb989dff4f0 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:24.689717 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b23a61-3a91-4b78-b6e6-3cb989dff4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:24.736550 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-54b23a61-3a91-4b78-b6e6-3cb989dff4f0 None None] MFA Rules not processed for user `ad0781ce20e54c3a9b7c61e0cccf6918`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:24.738619 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-54b23a61-3a91-4b78-b6e6-3cb989dff4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:24.888699 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c1cf0-b638-4091-b5b4-d77dfc5527c5 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:24.890497 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c1cf0-b638-4091-b5b4-d77dfc5527c5 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:24.890945 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c1cf0-b638-4091-b5b4-d77dfc5527c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:24.901459 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 817/1638] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:24.924297 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:24.933441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-713c1cf0-b638-4091-b5b4-d77dfc5527c5 None None] MFA Rules not processed for user `ad0781ce20e54c3a9b7c61e0cccf6918`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:24.934929 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-713c1cf0-b638-4091-b5b4-d77dfc5527c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:24.944069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 822/1639] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:24.955531 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b23a61-3a91-4b78-b6e6-3cb989dff4f0 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:24.981841 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b23a61-3a91-4b78-b6e6-3cb989dff4f0 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:24.991155 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b124ca2a-628c-4e91-82ce-a1cf9b7d0301 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:24.991738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124ca2a-628c-4e91-82ce-a1cf9b7d0301 service nova] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:24.992028 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124ca2a-628c-4e91-82ce-a1cf9b7d0301 service nova] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:24.992332 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124ca2a-628c-4e91-82ce-a1cf9b7d0301 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:25.073959 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b124ca2a-628c-4e91-82ce-a1cf9b7d0301 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:25.076395 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b124ca2a-628c-4e91-82ce-a1cf9b7d0301 service nova] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:25.107323 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 818/1640] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:24 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:25.740457 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bb083-fedc-47ea-9306-78578d2d2b4d None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:25.740755 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bb083-fedc-47ea-9306-78578d2d2b4d None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:25.741760 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bb083-fedc-47ea-9306-78578d2d2b4d None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:25.744453 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 823/1641] 10.222.0.20 () {54 vars in 912 bytes} [Wed Jul 6 13:45:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:45:25.750047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693b1d6-9215-45f8-93c6-7f85435d967e None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:25.751153 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693b1d6-9215-45f8-93c6-7f85435d967e None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:25.751589 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693b1d6-9215-45f8-93c6-7f85435d967e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:25.854914 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-a693b1d6-9215-45f8-93c6-7f85435d967e None None] MFA Rules not processed for user `b9f250f5dbc54d2e9293962ece848f6c`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:25.856594 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a693b1d6-9215-45f8-93c6-7f85435d967e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:25.963085 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 819/1642] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Jul 6 13:45:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:26.121445 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433bb083-fedc-47ea-9306-78578d2d2b4d None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:26.138267 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433bb083-fedc-47ea-9306-78578d2d2b4d None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:26.147434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ab9473-bf5d-4ede-aec0-2a76a13de111 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:26.150461 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab9473-bf5d-4ede-aec0-2a76a13de111 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:26.150763 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab9473-bf5d-4ede-aec0-2a76a13de111 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:26.151054 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab9473-bf5d-4ede-aec0-2a76a13de111 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:26.219933 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ab9473-bf5d-4ede-aec0-2a76a13de111 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:26.235619 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ab9473-bf5d-4ede-aec0-2a76a13de111 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:26.265282 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 824/1643] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:27.061809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950156e-8227-46f9-a83b-55ecce199c1d None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:27.062316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950156e-8227-46f9-a83b-55ecce199c1d None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:27.062386 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950156e-8227-46f9-a83b-55ecce199c1d None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:27.064315 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 820/1644] 10.222.0.20 () {54 vars in 960 bytes} [Wed Jul 6 13:45:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:45:27.070761 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44872488-61fc-462e-9c2f-18e2f3acf671 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:27.071481 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44872488-61fc-462e-9c2f-18e2f3acf671 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:27.071786 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44872488-61fc-462e-9c2f-18e2f3acf671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:27.121418 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-44872488-61fc-462e-9c2f-18e2f3acf671 None None] MFA Rules not processed for user `890d2ae150d647a18dae25159cf97484`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:27.122574 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-44872488-61fc-462e-9c2f-18e2f3acf671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:27.315753 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 825/1645] 10.222.0.20 () {58 vars in 1057 bytes} [Wed Jul 6 13:45:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:27.325194 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950156e-8227-46f9-a83b-55ecce199c1d None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:27.355781 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950156e-8227-46f9-a83b-55ecce199c1d None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:27.392810 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96365297-8f2b-4d94-90e9-bfa3abf3ef45 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '890d2ae150d647a18dae25159cf97484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:27.403910 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96365297-8f2b-4d94-90e9-bfa3abf3ef45 service placement] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:27.404197 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96365297-8f2b-4d94-90e9-bfa3abf3ef45 service placement] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:27.404486 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96365297-8f2b-4d94-90e9-bfa3abf3ef45 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:27.409800 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-96365297-8f2b-4d94-90e9-bfa3abf3ef45 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:27.454763 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96365297-8f2b-4d94-90e9-bfa3abf3ef45 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:27.456487 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96365297-8f2b-4d94-90e9-bfa3abf3ef45 service placement] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:27.481781 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 821/1646] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:28.507267 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44872488-61fc-462e-9c2f-18e2f3acf671 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:28.650077 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb406331-0151-44dd-88be-8e2b3565be41 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:28.650962 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb406331-0151-44dd-88be-8e2b3565be41 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:28.651427 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb406331-0151-44dd-88be-8e2b3565be41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:28.686010 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44872488-61fc-462e-9c2f-18e2f3acf671 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:28.697980 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaea931-8e46-4236-95e7-a4a3ff17a1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:28.699128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaea931-8e46-4236-95e7-a4a3ff17a1ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:28.699569 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaea931-8e46-4236-95e7-a4a3ff17a1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:28.700072 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaea931-8e46-4236-95e7-a4a3ff17a1ca admin admin] PATH_INFO: `/v3/users/3b07c60a91d241c6b1a42bc32328c274` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:28.710172 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaea931-8e46-4236-95e7-a4a3ff17a1ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b07c60a91d241c6b1a42bc32328c274)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:28.711455 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: WARNING py.warnings [None req-4eaea931-8e46-4236-95e7-a4a3ff17a1ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 13:45:28.711455 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: warnings.warn(msg) Jul 06 13:45:28.711455 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: Jul 06 13:45:28.713171 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaea931-8e46-4236-95e7-a4a3ff17a1ca admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:28.734189 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-fb406331-0151-44dd-88be-8e2b3565be41 None None] MFA Rules not processed for user `7b572a8f465048558fc969b85b2aec5b`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:28.736669 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-fb406331-0151-44dd-88be-8e2b3565be41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:28.880387 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:28.896701 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-4eaea931-8e46-4236-95e7-a4a3ff17a1ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b07c60a91d241c6b1a42bc32328c274'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:28.935923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-4eaea931-8e46-4236-95e7-a4a3ff17a1ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b07c60a91d241c6b1a42bc32328c274'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:29.002563 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-4eaea931-8e46-4236-95e7-a4a3ff17a1ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b07c60a91d241c6b1a42bc32328c274'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:29.006255 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-4eaea931-8e46-4236-95e7-a4a3ff17a1ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b07c60a91d241c6b1a42bc32328c274'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:29.049911 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 822/1647] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:29.066255 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb406331-0151-44dd-88be-8e2b3565be41 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:29.098731 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb406331-0151-44dd-88be-8e2b3565be41 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:29.125228 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b0b893-fa0b-4644-b397-7ae2b6df9d43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:29.129484 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0b893-fa0b-4644-b397-7ae2b6df9d43 service nova] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:29.129976 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0b893-fa0b-4644-b397-7ae2b6df9d43 service nova] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:29.131995 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0b893-fa0b-4644-b397-7ae2b6df9d43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:29.133057 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-4eaea931-8e46-4236-95e7-a4a3ff17a1ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b07c60a91d241c6b1a42bc32328c274'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:29.159529 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-56b0b893-fa0b-4644-b397-7ae2b6df9d43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:29.164885 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 826/1648] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:45:28 2022] DELETE /identity/v3/users/3b07c60a91d241c6b1a42bc32328c274 => generated 0 bytes in 658 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:29.173095 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9b09a-3d7f-48e8-9830-3d175c2325e2 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:29.173632 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9b09a-3d7f-48e8-9830-3d175c2325e2 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:29.175483 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9b09a-3d7f-48e8-9830-3d175c2325e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:29.204645 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:29.232613 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-80c9b09a-3d7f-48e8-9830-3d175c2325e2 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:29.235442 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-80c9b09a-3d7f-48e8-9830-3d175c2325e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:29.295961 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b0b893-fa0b-4644-b397-7ae2b6df9d43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:29.326821 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b0b893-fa0b-4644-b397-7ae2b6df9d43 service nova] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:29.370849 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 827/1649] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:29.389906 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9b09a-3d7f-48e8-9830-3d175c2325e2 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:29.402773 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-80c9b09a-3d7f-48e8-9830-3d175c2325e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:29.404058 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 823/1650] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:29.638196 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9b09a-3d7f-48e8-9830-3d175c2325e2 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:29.657158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f16c5-81f5-43d1-9696-3e1e26afbe13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:29.658377 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f16c5-81f5-43d1-9696-3e1e26afbe13 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:29.658844 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f16c5-81f5-43d1-9696-3e1e26afbe13 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:29.659298 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f16c5-81f5-43d1-9696-3e1e26afbe13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:29.691040 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f16c5-81f5-43d1-9696-3e1e26afbe13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:29.693337 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f16c5-81f5-43d1-9696-3e1e26afbe13 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:29.706541 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 828/1651] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:29.912373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafa2a9-87e5-48a6-91a1-ad8e9c76cda8 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:29.913185 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafa2a9-87e5-48a6-91a1-ad8e9c76cda8 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:29.913185 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafa2a9-87e5-48a6-91a1-ad8e9c76cda8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:29.982736 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-adafa2a9-87e5-48a6-91a1-ad8e9c76cda8 None None] MFA Rules not processed for user `3d7cacb73c8342579d9f990b0193487c`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:29.984108 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-adafa2a9-87e5-48a6-91a1-ad8e9c76cda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:30.169066 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 824/1652] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:30.180795 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f16c5-81f5-43d1-9696-3e1e26afbe13 service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:30.182381 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-b90f16c5-81f5-43d1-9696-3e1e26afbe13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:30.376795 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f16c5-81f5-43d1-9696-3e1e26afbe13 service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:30.387759 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac06a6-3051-4f7c-b283-f0e491fd53cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:30.388442 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac06a6-3051-4f7c-b283-f0e491fd53cd service nova] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:30.389401 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac06a6-3051-4f7c-b283-f0e491fd53cd service nova] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:30.389401 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac06a6-3051-4f7c-b283-f0e491fd53cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:30.400683 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac06a6-3051-4f7c-b283-f0e491fd53cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:30.402950 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac06a6-3051-4f7c-b283-f0e491fd53cd service nova] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:30.421408 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 829/1653] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:30.951876 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7601fb9f-c3df-4007-8e45-fb0ce0c9af0f None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:30.952450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7601fb9f-c3df-4007-8e45-fb0ce0c9af0f None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:30.952533 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7601fb9f-c3df-4007-8e45-fb0ce0c9af0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:31.019132 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-7601fb9f-c3df-4007-8e45-fb0ce0c9af0f None None] MFA Rules not processed for user `0c83c05bec4f456d8aa7d51732ddc8af`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:31.020357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-7601fb9f-c3df-4007-8e45-fb0ce0c9af0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:31.157560 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 825/1654] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:31.169284 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0cc38-86d2-4c95-84c7-95aae50db7e2 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:31.170288 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0cc38-86d2-4c95-84c7-95aae50db7e2 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:31.170718 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0cc38-86d2-4c95-84c7-95aae50db7e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:31.198454 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-c2b0cc38-86d2-4c95-84c7-95aae50db7e2 None None] MFA Rules not processed for user `0c83c05bec4f456d8aa7d51732ddc8af`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:31.200738 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-c2b0cc38-86d2-4c95-84c7-95aae50db7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:31.215024 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 830/1655] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:31.226282 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7601fb9f-c3df-4007-8e45-fb0ce0c9af0f None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:31.242009 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7601fb9f-c3df-4007-8e45-fb0ce0c9af0f None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:31.267898 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352f8e04-168b-44aa-b283-29f6d6dbe2bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:31.268902 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f8e04-168b-44aa-b283-29f6d6dbe2bb service nova] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:31.269291 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f8e04-168b-44aa-b283-29f6d6dbe2bb service nova] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:31.269567 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f8e04-168b-44aa-b283-29f6d6dbe2bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:31.280338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352f8e04-168b-44aa-b283-29f6d6dbe2bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:31.282454 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352f8e04-168b-44aa-b283-29f6d6dbe2bb service nova] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:31.311777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 826/1656] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:31 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:31.317870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b0cc38-86d2-4c95-84c7-95aae50db7e2 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:31.319431 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-c2b0cc38-86d2-4c95-84c7-95aae50db7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:31.342116 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b0cc38-86d2-4c95-84c7-95aae50db7e2 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:31.353464 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712929b2-c590-4df9-8652-f7e6cd8242e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:31.355230 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712929b2-c590-4df9-8652-f7e6cd8242e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:31.356844 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712929b2-c590-4df9-8652-f7e6cd8242e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:31.361592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712929b2-c590-4df9-8652-f7e6cd8242e7 admin admin] PATH_INFO: `/v3/projects/c4e36049a0fd4b12a1ab55b5b4d5adfe` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:31.363365 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712929b2-c590-4df9-8652-f7e6cd8242e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4e36049a0fd4b12a1ab55b5b4d5adfe)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:31.369896 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712929b2-c590-4df9-8652-f7e6cd8242e7 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:31.431967 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:31.465470 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-712929b2-c590-4df9-8652-f7e6cd8242e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4e36049a0fd4b12a1ab55b5b4d5adfe', 'request_id': 'req-712929b2-c590-4df9-8652-f7e6cd8242e7'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:31.485897 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-712929b2-c590-4df9-8652-f7e6cd8242e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4e36049a0fd4b12a1ab55b5b4d5adfe', 'request_id': 'req-712929b2-c590-4df9-8652-f7e6cd8242e7'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:31.499150 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-712929b2-c590-4df9-8652-f7e6cd8242e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4e36049a0fd4b12a1ab55b5b4d5adfe', 'request_id': 'req-712929b2-c590-4df9-8652-f7e6cd8242e7'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:31.516464 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-712929b2-c590-4df9-8652-f7e6cd8242e7 admin admin] The token cache is being invalidate because project c4e36049a0fd4b12a1ab55b5b4d5adfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:45:31.516873 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-712929b2-c590-4df9-8652-f7e6cd8242e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:31.517796 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-712929b2-c590-4df9-8652-f7e6cd8242e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:31.519472 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 831/1657] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:45:31 2022] DELETE /identity/v3/projects/c4e36049a0fd4b12a1ab55b5b4d5adfe => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:31.524577 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60133ce5-b042-4143-9082-52f6c6d3c40d None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:31.524916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60133ce5-b042-4143-9082-52f6c6d3c40d None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:31.525229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60133ce5-b042-4143-9082-52f6c6d3c40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:31.590020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-60133ce5-b042-4143-9082-52f6c6d3c40d None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:31.591243 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-60133ce5-b042-4143-9082-52f6c6d3c40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:31.687754 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:31.785464 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 827/1658] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:31.788341 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f190e1-bef0-466e-a23a-cc4ff3ee9a65 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:31.788643 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f190e1-bef0-466e-a23a-cc4ff3ee9a65 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:31.788957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f190e1-bef0-466e-a23a-cc4ff3ee9a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:31.851264 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-70f190e1-bef0-466e-a23a-cc4ff3ee9a65 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:31.854610 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-70f190e1-bef0-466e-a23a-cc4ff3ee9a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:31.865756 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 832/1659] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:31.871321 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60133ce5-b042-4143-9082-52f6c6d3c40d None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:31.882870 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60133ce5-b042-4143-9082-52f6c6d3c40d None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:31.891273 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2541d-5340-4a68-a881-7333e844543c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:31.892368 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2541d-5340-4a68-a881-7333e844543c admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:31.892862 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2541d-5340-4a68-a881-7333e844543c admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:31.893385 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2541d-5340-4a68-a881-7333e844543c admin admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:31.894397 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d2541d-5340-4a68-a881-7333e844543c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:45:31.894904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d2541d-5340-4a68-a881-7333e844543c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:31.898100 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d2541d-5340-4a68-a881-7333e844543c admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:31.921896 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 828/1660] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:45:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:45:31.926544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f190e1-bef0-466e-a23a-cc4ff3ee9a65 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:31.940709 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f190e1-bef0-466e-a23a-cc4ff3ee9a65 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:31.951863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3426f16-681a-4839-80b8-1b8cfafb7e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:31.952679 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3426f16-681a-4839-80b8-1b8cfafb7e01 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:31.953031 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3426f16-681a-4839-80b8-1b8cfafb7e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:31.953319 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3426f16-681a-4839-80b8-1b8cfafb7e01 admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:31.954159 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3426f16-681a-4839-80b8-1b8cfafb7e01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:31.958101 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3426f16-681a-4839-80b8-1b8cfafb7e01 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:32.020276 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 833/1661] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:45:31 2022] POST /identity/v3/projects => generated 373 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:32.021479 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2541d-5340-4a68-a881-7333e844543c admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:32.037878 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2541d-5340-4a68-a881-7333e844543c admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:32.240400 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5109c-90f6-4bbf-8fa0-9b76c14c6f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:32.241308 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5109c-90f6-4bbf-8fa0-9b76c14c6f45 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:32.241598 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5109c-90f6-4bbf-8fa0-9b76c14c6f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:32.241876 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5109c-90f6-4bbf-8fa0-9b76c14c6f45 admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:32.243595 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5109c-90f6-4bbf-8fa0-9b76c14c6f45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:32.245745 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5109c-90f6-4bbf-8fa0-9b76c14c6f45 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:32.323204 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3426f16-681a-4839-80b8-1b8cfafb7e01 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:32.325055 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 829/1662] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:45:32 2022] POST /identity/v3/users => generated 287 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:45:32.334411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3426f16-681a-4839-80b8-1b8cfafb7e01 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:32.347168 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6576d353-3b9c-42d3-bfaa-5c60bbe0106e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:32.348037 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576d353-3b9c-42d3-bfaa-5c60bbe0106e admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:32.348461 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576d353-3b9c-42d3-bfaa-5c60bbe0106e admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:32.349035 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576d353-3b9c-42d3-bfaa-5c60bbe0106e admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:32.350366 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6576d353-3b9c-42d3-bfaa-5c60bbe0106e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:32.352819 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6576d353-3b9c-42d3-bfaa-5c60bbe0106e admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:32.386030 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 834/1663] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:32.386504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5109c-90f6-4bbf-8fa0-9b76c14c6f45 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:32.405577 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5109c-90f6-4bbf-8fa0-9b76c14c6f45 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:32.425350 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e65e7a-8aed-4084-b8b2-a1462d56afd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:32.425350 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e65e7a-8aed-4084-b8b2-a1462d56afd4 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:32.425350 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e65e7a-8aed-4084-b8b2-a1462d56afd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:32.425350 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e65e7a-8aed-4084-b8b2-a1462d56afd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:32.427224 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e65e7a-8aed-4084-b8b2-a1462d56afd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:32.429262 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e65e7a-8aed-4084-b8b2-a1462d56afd4 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:32.451224 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 830/1664] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:45:32.455823 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6576d353-3b9c-42d3-bfaa-5c60bbe0106e admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:32.484376 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6576d353-3b9c-42d3-bfaa-5c60bbe0106e admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:32.496964 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3fd840-0983-43f4-abb5-554cd127d8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:32.499060 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fd840-0983-43f4-abb5-554cd127d8bc admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:32.500227 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fd840-0983-43f4-abb5-554cd127d8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:32.500689 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fd840-0983-43f4-abb5-554cd127d8bc admin admin] PATH_INFO: `/v3/projects/eceed8c4586040e78e2514434c8451ba/users/41e942a600e74f97b6b09c9202c4a636/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:32.531419 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3fd840-0983-43f4-abb5-554cd127d8bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=eceed8c4586040e78e2514434c8451ba, user_id=41e942a600e74f97b6b09c9202c4a636, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:32.534523 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3fd840-0983-43f4-abb5-554cd127d8bc admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:32.569886 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 835/1665] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:45:32 2022] PUT /identity/v3/projects/eceed8c4586040e78e2514434c8451ba/users/41e942a600e74f97b6b09c9202c4a636/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:32.580809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeea4d4-018b-4060-bb1f-00324c9cf905 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:32.581282 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeea4d4-018b-4060-bb1f-00324c9cf905 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:32.581571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeea4d4-018b-4060-bb1f-00324c9cf905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:32.696877 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-6eeea4d4-018b-4060-bb1f-00324c9cf905 None None] MFA Rules not processed for user `41e942a600e74f97b6b09c9202c4a636`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:32.698140 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-6eeea4d4-018b-4060-bb1f-00324c9cf905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:32.860423 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da9325-5009-4311-b105-ee1d786fb3b7 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:32.860914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da9325-5009-4311-b105-ee1d786fb3b7 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:32.861622 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da9325-5009-4311-b105-ee1d786fb3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:32.879200 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 831/1666] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:32.889882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-b8da9325-5009-4311-b105-ee1d786fb3b7 None None] MFA Rules not processed for user `0c83c05bec4f456d8aa7d51732ddc8af`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:32.891769 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-b8da9325-5009-4311-b105-ee1d786fb3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:32.892935 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeea4d4-018b-4060-bb1f-00324c9cf905 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:32.894734 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-6eeea4d4-018b-4060-bb1f-00324c9cf905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:33.009338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:33.075674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 836/1667] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:45:33.119912 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeea4d4-018b-4060-bb1f-00324c9cf905 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:33.129591 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d36597c-496f-4cec-b6e0-5c3819f1e02d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:33.130863 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36597c-496f-4cec-b6e0-5c3819f1e02d service nova] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:33.131430 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36597c-496f-4cec-b6e0-5c3819f1e02d service nova] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:33.131843 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36597c-496f-4cec-b6e0-5c3819f1e02d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:33.142356 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d36597c-496f-4cec-b6e0-5c3819f1e02d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:33.144150 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d36597c-496f-4cec-b6e0-5c3819f1e02d service nova] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:33.156368 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 832/1668] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:33.603664 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8da9325-5009-4311-b105-ee1d786fb3b7 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:33.863530 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:33.910658 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8da9325-5009-4311-b105-ee1d786fb3b7 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:33.946751 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214669ca-930b-4710-be75-119a72fab31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:33.947600 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214669ca-930b-4710-be75-119a72fab31f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:33.947884 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214669ca-930b-4710-be75-119a72fab31f admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:33.948161 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214669ca-930b-4710-be75-119a72fab31f admin admin] PATH_INFO: `/v3/users/41e942a600e74f97b6b09c9202c4a636` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:33.957533 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214669ca-930b-4710-be75-119a72fab31f admin admin] RBAC: Authorizing `identity:delete_user(user_id=41e942a600e74f97b6b09c9202c4a636)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:33.959500 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214669ca-930b-4710-be75-119a72fab31f admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:34.105245 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-214669ca-930b-4710-be75-119a72fab31f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41e942a600e74f97b6b09c9202c4a636'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:34.129250 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-214669ca-930b-4710-be75-119a72fab31f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41e942a600e74f97b6b09c9202c4a636'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:34.154010 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-214669ca-930b-4710-be75-119a72fab31f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41e942a600e74f97b6b09c9202c4a636'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:34.155946 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-214669ca-930b-4710-be75-119a72fab31f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41e942a600e74f97b6b09c9202c4a636'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:34.218430 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-214669ca-930b-4710-be75-119a72fab31f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41e942a600e74f97b6b09c9202c4a636'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:34.235521 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 837/1669] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:45:33 2022] DELETE /identity/v3/users/41e942a600e74f97b6b09c9202c4a636 => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:34.241243 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b2223-291c-41f5-b981-6d2bd75b8d9d None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:34.242505 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b2223-291c-41f5-b981-6d2bd75b8d9d None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:34.243503 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b2223-291c-41f5-b981-6d2bd75b8d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:34.298049 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-e63b2223-291c-41f5-b981-6d2bd75b8d9d None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:34.302962 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-e63b2223-291c-41f5-b981-6d2bd75b8d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:34.692353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 833/1670] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 450 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:34.698714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957e2d7-92c0-4198-b846-0d73e0f8027c None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:34.699320 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957e2d7-92c0-4198-b846-0d73e0f8027c None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:34.699320 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957e2d7-92c0-4198-b846-0d73e0f8027c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:34.716725 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b2223-291c-41f5-b981-6d2bd75b8d9d None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:34.717316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-e63b2223-291c-41f5-b981-6d2bd75b8d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:34.735001 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b2223-291c-41f5-b981-6d2bd75b8d9d None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:34.741489 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a92d0-acbd-430f-a6ef-0df27f56c1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:34.743410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a92d0-acbd-430f-a6ef-0df27f56c1e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:34.743829 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a92d0-acbd-430f-a6ef-0df27f56c1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:34.744404 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a92d0-acbd-430f-a6ef-0df27f56c1e1 admin admin] PATH_INFO: `/v3/users/ae1587d0dc6246f9997824193eef6d78` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:34.745949 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a92d0-acbd-430f-a6ef-0df27f56c1e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae1587d0dc6246f9997824193eef6d78)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:34.749156 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a92d0-acbd-430f-a6ef-0df27f56c1e1 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:34.786832 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-7957e2d7-92c0-4198-b846-0d73e0f8027c None None] MFA Rules not processed for user `c4535134181e489fb532f617264d5855`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:34.796533 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-7957e2d7-92c0-4198-b846-0d73e0f8027c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:34.903391 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-489a92d0-acbd-430f-a6ef-0df27f56c1e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae1587d0dc6246f9997824193eef6d78'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:34.938272 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-489a92d0-acbd-430f-a6ef-0df27f56c1e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae1587d0dc6246f9997824193eef6d78'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:34.940188 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-489a92d0-acbd-430f-a6ef-0df27f56c1e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae1587d0dc6246f9997824193eef6d78'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:34.957498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-489a92d0-acbd-430f-a6ef-0df27f56c1e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae1587d0dc6246f9997824193eef6d78'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:34.983479 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-489a92d0-acbd-430f-a6ef-0df27f56c1e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae1587d0dc6246f9997824193eef6d78'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:35.003755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 834/1671] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:45:34 2022] DELETE /identity/v3/users/ae1587d0dc6246f9997824193eef6d78 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:35.009338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a92d0-acbd-430f-a6ef-0df27f56c1e1 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:35.014293 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-489a92d0-acbd-430f-a6ef-0df27f56c1e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:35.090306 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 838/1672] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Jul 6 13:45:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:35.091925 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197b1d3-3a34-4861-9ed9-b0aa95ff2c5d None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:35.095554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197b1d3-3a34-4861-9ed9-b0aa95ff2c5d None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:35.109651 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197b1d3-3a34-4861-9ed9-b0aa95ff2c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:35.171668 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:35.208867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-2197b1d3-3a34-4861-9ed9-b0aa95ff2c5d None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:35.212803 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-2197b1d3-3a34-4861-9ed9-b0aa95ff2c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:35.350067 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a92d0-acbd-430f-a6ef-0df27f56c1e1 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:35.364775 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d541b2c-ff7b-4e4c-a99e-f2df42de1203 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:35.366182 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d541b2c-ff7b-4e4c-a99e-f2df42de1203 service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:35.366681 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d541b2c-ff7b-4e4c-a99e-f2df42de1203 service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:35.367165 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d541b2c-ff7b-4e4c-a99e-f2df42de1203 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:35.371271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-7d541b2c-ff7b-4e4c-a99e-f2df42de1203 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:35.575628 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d541b2c-ff7b-4e4c-a99e-f2df42de1203 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:35.577447 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d541b2c-ff7b-4e4c-a99e-f2df42de1203 service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:35.605312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 839/1673] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 514 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:35.606284 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197b1d3-3a34-4861-9ed9-b0aa95ff2c5d None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:35.615848 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-2197b1d3-3a34-4861-9ed9-b0aa95ff2c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:35.620743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 835/1674] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:35.670794 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:35.673592 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d541b2c-ff7b-4e4c-a99e-f2df42de1203 service neutron] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:35.675893 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-7d541b2c-ff7b-4e4c-a99e-f2df42de1203 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:35.700784 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d541b2c-ff7b-4e4c-a99e-f2df42de1203 service neutron] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:35.713858 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75269930-cddb-4fae-a0d9-a01c58149250 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:35.714905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75269930-cddb-4fae-a0d9-a01c58149250 service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:35.715449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75269930-cddb-4fae-a0d9-a01c58149250 service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:35.715898 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75269930-cddb-4fae-a0d9-a01c58149250 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:35.740068 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75269930-cddb-4fae-a0d9-a01c58149250 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:35.745124 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75269930-cddb-4fae-a0d9-a01c58149250 service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:35.798378 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 836/1675] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:35.802253 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e09a7f-3bbf-43a3-a31a-94857d0e7604 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:35.802589 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e09a7f-3bbf-43a3-a31a-94857d0e7604 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:35.806135 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e09a7f-3bbf-43a3-a31a-94857d0e7604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:35.829646 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197b1d3-3a34-4861-9ed9-b0aa95ff2c5d None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:35.841897 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76344b1f-c84b-4648-9e42-498266bddb1a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c1bea05c37646b0a05fa8dda0367ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:35.842578 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76344b1f-c84b-4648-9e42-498266bddb1a service swift] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:35.842867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76344b1f-c84b-4648-9e42-498266bddb1a service swift] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:35.843148 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76344b1f-c84b-4648-9e42-498266bddb1a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:35.875029 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-b7e09a7f-3bbf-43a3-a31a-94857d0e7604 None None] MFA Rules not processed for user `3d7cacb73c8342579d9f990b0193487c`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:35.893385 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-b7e09a7f-3bbf-43a3-a31a-94857d0e7604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:35.970624 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76344b1f-c84b-4648-9e42-498266bddb1a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:35.983547 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76344b1f-c84b-4648-9e42-498266bddb1a service swift] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:36.019113 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 840/1676] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jul 6 13:45:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 13:45:36.248059 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 837/1677] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 449 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:37.820426 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63536762-c94c-496e-801b-c8e645c9e18d None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:37.821621 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63536762-c94c-496e-801b-c8e645c9e18d None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:37.822053 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63536762-c94c-496e-801b-c8e645c9e18d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:37.859910 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-63536762-c94c-496e-801b-c8e645c9e18d None None] MFA Rules not processed for user `7b572a8f465048558fc969b85b2aec5b`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:37.863186 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-63536762-c94c-496e-801b-c8e645c9e18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:38.179813 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 841/1678] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:38.220612 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e09a7f-3bbf-43a3-a31a-94857d0e7604 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:38.222935 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-b7e09a7f-3bbf-43a3-a31a-94857d0e7604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:38.278205 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e09a7f-3bbf-43a3-a31a-94857d0e7604 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:38.294302 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be536903-007c-499b-9646-88ba822b0851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:38.294302 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be536903-007c-499b-9646-88ba822b0851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:38.294599 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be536903-007c-499b-9646-88ba822b0851 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:38.294810 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be536903-007c-499b-9646-88ba822b0851 admin admin] PATH_INFO: `/v3/projects/5080a0e6cfb24d799c3b31cec1d073bc` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:38.302786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be536903-007c-499b-9646-88ba822b0851 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5080a0e6cfb24d799c3b31cec1d073bc)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:38.302786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: WARNING py.warnings [None req-be536903-007c-499b-9646-88ba822b0851 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 13:45:38.302786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: warnings.warn(msg) Jul 06 13:45:38.302786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: Jul 06 13:45:38.302786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be536903-007c-499b-9646-88ba822b0851 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:38.468365 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-be536903-007c-499b-9646-88ba822b0851 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5080a0e6cfb24d799c3b31cec1d073bc', 'request_id': 'req-be536903-007c-499b-9646-88ba822b0851'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:38.506011 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-be536903-007c-499b-9646-88ba822b0851 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5080a0e6cfb24d799c3b31cec1d073bc', 'request_id': 'req-be536903-007c-499b-9646-88ba822b0851'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:38.518302 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:38.529909 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-be536903-007c-499b-9646-88ba822b0851 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5080a0e6cfb24d799c3b31cec1d073bc', 'request_id': 'req-be536903-007c-499b-9646-88ba822b0851'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:38.586954 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-be536903-007c-499b-9646-88ba822b0851 admin admin] The token cache is being invalidate because project 5080a0e6cfb24d799c3b31cec1d073bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:45:38.587918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-be536903-007c-499b-9646-88ba822b0851 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:38.589320 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-be536903-007c-499b-9646-88ba822b0851 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:38.597072 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 838/1679] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:45:38 2022] DELETE /identity/v3/projects/5080a0e6cfb24d799c3b31cec1d073bc => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:38.598355 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703372ff-34cd-450b-9242-a190f6b5049c None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:38.598698 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703372ff-34cd-450b-9242-a190f6b5049c None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:38.598992 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703372ff-34cd-450b-9242-a190f6b5049c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:38.674476 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-703372ff-34cd-450b-9242-a190f6b5049c None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:38.676309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-703372ff-34cd-450b-9242-a190f6b5049c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:38.845228 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:38.876497 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67a17b-811c-4526-8222-c8294acc5d26 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:38.879837 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67a17b-811c-4526-8222-c8294acc5d26 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:38.880121 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67a17b-811c-4526-8222-c8294acc5d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:38.883227 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 842/1680] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:45:38.919753 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-1a67a17b-811c-4526-8222-c8294acc5d26 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:38.922397 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-1a67a17b-811c-4526-8222-c8294acc5d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:38.957100 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703372ff-34cd-450b-9242-a190f6b5049c None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:38.963857 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 839/1681] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:45:38.973149 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703372ff-34cd-450b-9242-a190f6b5049c None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:38.987987 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71beb80-b8f3-4812-8cee-e3b61f243222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:38.989028 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71beb80-b8f3-4812-8cee-e3b61f243222 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:38.990321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71beb80-b8f3-4812-8cee-e3b61f243222 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:38.991597 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71beb80-b8f3-4812-8cee-e3b61f243222 admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:38.992622 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71beb80-b8f3-4812-8cee-e3b61f243222 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:45:38.994316 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71beb80-b8f3-4812-8cee-e3b61f243222 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:38.998771 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71beb80-b8f3-4812-8cee-e3b61f243222 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:39.040713 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67a17b-811c-4526-8222-c8294acc5d26 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:39.047754 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 843/1682] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:45:39.067918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67a17b-811c-4526-8222-c8294acc5d26 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:39.081791 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec13bb-ff3e-47cd-a47c-73d5cc40b142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:39.082674 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec13bb-ff3e-47cd-a47c-73d5cc40b142 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:39.085264 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec13bb-ff3e-47cd-a47c-73d5cc40b142 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:39.085796 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec13bb-ff3e-47cd-a47c-73d5cc40b142 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:39.088888 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec13bb-ff3e-47cd-a47c-73d5cc40b142 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:39.092306 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec13bb-ff3e-47cd-a47c-73d5cc40b142 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:39.161777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 840/1683] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:45:39 2022] POST /identity/v3/projects => generated 405 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:39.166963 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71beb80-b8f3-4812-8cee-e3b61f243222 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:39.186703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71beb80-b8f3-4812-8cee-e3b61f243222 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:39.387758 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec0271-5243-4844-badb-af61fb57d30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:39.388735 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec0271-5243-4844-badb-af61fb57d30a admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:39.389334 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec0271-5243-4844-badb-af61fb57d30a admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:39.389806 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec0271-5243-4844-badb-af61fb57d30a admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:39.391411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caec0271-5243-4844-badb-af61fb57d30a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:39.393776 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caec0271-5243-4844-badb-af61fb57d30a admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:39.473454 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 844/1684] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:45:39 2022] POST /identity/v3/users => generated 303 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:39.478068 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec13bb-ff3e-47cd-a47c-73d5cc40b142 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:39.506894 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec13bb-ff3e-47cd-a47c-73d5cc40b142 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:39.516983 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec348f9-dc60-4254-a8fb-aba9e7fdff02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:39.517677 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec348f9-dc60-4254-a8fb-aba9e7fdff02 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:39.517959 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec348f9-dc60-4254-a8fb-aba9e7fdff02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:39.518236 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec348f9-dc60-4254-a8fb-aba9e7fdff02 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:39.518969 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec348f9-dc60-4254-a8fb-aba9e7fdff02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:39.520698 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec348f9-dc60-4254-a8fb-aba9e7fdff02 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:39.527845 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec0271-5243-4844-badb-af61fb57d30a admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:39.529941 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-caec0271-5243-4844-badb-af61fb57d30a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:39.557614 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec0271-5243-4844-badb-af61fb57d30a admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:39.570053 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 841/1685] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:39.571737 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec348f9-dc60-4254-a8fb-aba9e7fdff02 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:39.572419 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230c777-8b5d-4009-8f58-6ccb5315cd60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:39.573559 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230c777-8b5d-4009-8f58-6ccb5315cd60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:39.573844 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230c777-8b5d-4009-8f58-6ccb5315cd60 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:39.574125 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230c777-8b5d-4009-8f58-6ccb5315cd60 admin admin] PATH_INFO: `/v3/projects/eceed8c4586040e78e2514434c8451ba` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:39.584880 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230c777-8b5d-4009-8f58-6ccb5315cd60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eceed8c4586040e78e2514434c8451ba)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:39.592708 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230c777-8b5d-4009-8f58-6ccb5315cd60 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:39.596991 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec348f9-dc60-4254-a8fb-aba9e7fdff02 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:39.604424 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745985d1-4b08-43e7-ab0e-6e95ada646a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:39.605554 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745985d1-4b08-43e7-ab0e-6e95ada646a2 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:39.606073 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745985d1-4b08-43e7-ab0e-6e95ada646a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:39.606690 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745985d1-4b08-43e7-ab0e-6e95ada646a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:39.607705 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745985d1-4b08-43e7-ab0e-6e95ada646a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:39.613357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745985d1-4b08-43e7-ab0e-6e95ada646a2 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:39.685987 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 842/1686] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:39.687609 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745985d1-4b08-43e7-ab0e-6e95ada646a2 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:39.716357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745985d1-4b08-43e7-ab0e-6e95ada646a2 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:39.740150 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-6230c777-8b5d-4009-8f58-6ccb5315cd60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eceed8c4586040e78e2514434c8451ba', 'request_id': 'req-6230c777-8b5d-4009-8f58-6ccb5315cd60'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:39.751140 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-6230c777-8b5d-4009-8f58-6ccb5315cd60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eceed8c4586040e78e2514434c8451ba', 'request_id': 'req-6230c777-8b5d-4009-8f58-6ccb5315cd60'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:39.783889 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-6230c777-8b5d-4009-8f58-6ccb5315cd60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eceed8c4586040e78e2514434c8451ba', 'request_id': 'req-6230c777-8b5d-4009-8f58-6ccb5315cd60'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:39.822906 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-6230c777-8b5d-4009-8f58-6ccb5315cd60 admin admin] The token cache is being invalidate because project eceed8c4586040e78e2514434c8451ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:45:39.823360 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-6230c777-8b5d-4009-8f58-6ccb5315cd60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:39.825601 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-6230c777-8b5d-4009-8f58-6ccb5315cd60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:39.828183 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 845/1687] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:45:39 2022] DELETE /identity/v3/projects/eceed8c4586040e78e2514434c8451ba => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:39.833952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68403152-ac23-4185-a148-3e2f91186000 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:39.838193 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68403152-ac23-4185-a148-3e2f91186000 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:39.838934 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68403152-ac23-4185-a148-3e2f91186000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:39.870564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-68403152-ac23-4185-a148-3e2f91186000 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:39.873513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-68403152-ac23-4185-a148-3e2f91186000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:39.879789 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57077468-1a89-4270-a060-f498c414427b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:39.880718 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57077468-1a89-4270-a060-f498c414427b admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:39.881257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57077468-1a89-4270-a060-f498c414427b admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:39.881738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57077468-1a89-4270-a060-f498c414427b admin admin] PATH_INFO: `/v3/projects/fb70788b58f648acbdfe25cc67c11ace/users/1b405df0afb247808aea6737cadc14c7/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:39.889304 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 846/1688] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:39.897254 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84afdeed-2a8f-4b8e-97ae-5d8599e207dc None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:39.899627 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84afdeed-2a8f-4b8e-97ae-5d8599e207dc None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:39.901362 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84afdeed-2a8f-4b8e-97ae-5d8599e207dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:39.922883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57077468-1a89-4270-a060-f498c414427b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb70788b58f648acbdfe25cc67c11ace, user_id=1b405df0afb247808aea6737cadc14c7, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:39.926589 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57077468-1a89-4270-a060-f498c414427b admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:39.948876 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 843/1689] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:45:39 2022] PUT /identity/v3/projects/fb70788b58f648acbdfe25cc67c11ace/users/1b405df0afb247808aea6737cadc14c7/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:39.964954 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297021c9-1eec-432e-969b-b271ff5291e4 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:39.966417 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297021c9-1eec-432e-969b-b271ff5291e4 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:39.967033 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297021c9-1eec-432e-969b-b271ff5291e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:39.967994 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-84afdeed-2a8f-4b8e-97ae-5d8599e207dc None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:39.969802 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-84afdeed-2a8f-4b8e-97ae-5d8599e207dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:40.035860 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-297021c9-1eec-432e-969b-b271ff5291e4 None None] MFA Rules not processed for user `1b405df0afb247808aea6737cadc14c7`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:40.037307 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-297021c9-1eec-432e-969b-b271ff5291e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:40.069587 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:40.127871 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 847/1690] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:40.129708 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d203c36a-f415-4b7d-b40e-cc5defd836aa None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:40.130154 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d203c36a-f415-4b7d-b40e-cc5defd836aa None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:40.130572 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d203c36a-f415-4b7d-b40e-cc5defd836aa None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:40.140189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 848/1691] 10.222.0.20 () {54 vars in 912 bytes} [Wed Jul 6 13:45:40 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:45:40.142907 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d203c36a-f415-4b7d-b40e-cc5defd836aa None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:40.203015 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d203c36a-f415-4b7d-b40e-cc5defd836aa None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:40.227811 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10eb90c-9446-4a89-b904-3c4d5d49cfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:40.228672 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10eb90c-9446-4a89-b904-3c4d5d49cfb1 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:40.230162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10eb90c-9446-4a89-b904-3c4d5d49cfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:40.230632 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10eb90c-9446-4a89-b904-3c4d5d49cfb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:40.240038 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10eb90c-9446-4a89-b904-3c4d5d49cfb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:45:40.240439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10eb90c-9446-4a89-b904-3c4d5d49cfb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:40.245015 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10eb90c-9446-4a89-b904-3c4d5d49cfb1 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:40.257254 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 844/1692] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:39 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:40.259329 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54626ef-3c79-416d-ba3f-3988725cd665 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:40.259603 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54626ef-3c79-416d-ba3f-3988725cd665 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:40.259879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54626ef-3c79-416d-ba3f-3988725cd665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:40.324236 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 849/1693] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:45:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:45:40.324630 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-b54626ef-3c79-416d-ba3f-3988725cd665 None None] MFA Rules not processed for user `b9f250f5dbc54d2e9293962ece848f6c`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:40.326195 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-b54626ef-3c79-416d-ba3f-3988725cd665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:40.327980 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10eb90c-9446-4a89-b904-3c4d5d49cfb1 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:40.343104 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:40.345988 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-b10eb90c-9446-4a89-b904-3c4d5d49cfb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:40.448181 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10eb90c-9446-4a89-b904-3c4d5d49cfb1 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:40.465108 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 845/1694] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Jul 6 13:45:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:40.466142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54626ef-3c79-416d-ba3f-3988725cd665 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:40.478434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952af16-a2ec-45be-b1c7-dac49e617540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:40.478797 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54626ef-3c79-416d-ba3f-3988725cd665 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:40.479260 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952af16-a2ec-45be-b1c7-dac49e617540 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:40.479453 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952af16-a2ec-45be-b1c7-dac49e617540 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:40.479670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952af16-a2ec-45be-b1c7-dac49e617540 admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:40.480608 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0952af16-a2ec-45be-b1c7-dac49e617540 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:40.482833 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0952af16-a2ec-45be-b1c7-dac49e617540 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:40.484299 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b07b2-3bde-4197-a290-536337efbd0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:40.485536 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b07b2-3bde-4197-a290-536337efbd0a admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:40.485818 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b07b2-3bde-4197-a290-536337efbd0a admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:40.486093 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b07b2-3bde-4197-a290-536337efbd0a admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:40.486790 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b07b2-3bde-4197-a290-536337efbd0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:40.489547 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b07b2-3bde-4197-a290-536337efbd0a admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:40.552404 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 846/1695] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:45:40 2022] POST /identity/v3/projects => generated 365 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:40.563555 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b07b2-3bde-4197-a290-536337efbd0a admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:40.631097 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 850/1696] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:45:40 2022] POST /identity/v3/projects => generated 403 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:40.632383 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952af16-a2ec-45be-b1c7-dac49e617540 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:40.780819 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b07b2-3bde-4197-a290-536337efbd0a admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:40.811575 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952af16-a2ec-45be-b1c7-dac49e617540 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:40.829289 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325ffb5-a2ec-4707-9962-fdd250f843aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:40.830420 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325ffb5-a2ec-4707-9962-fdd250f843aa admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:40.830648 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325ffb5-a2ec-4707-9962-fdd250f843aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:40.831070 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325ffb5-a2ec-4707-9962-fdd250f843aa admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:40.831808 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6325ffb5-a2ec-4707-9962-fdd250f843aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:40.833845 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6325ffb5-a2ec-4707-9962-fdd250f843aa admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:40.900293 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 851/1697] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:45:40 2022] POST /identity/v3/users => generated 308 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:40.902664 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325ffb5-a2ec-4707-9962-fdd250f843aa admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:40.909168 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-6325ffb5-a2ec-4707-9962-fdd250f843aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:41.046678 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa114e9-abd9-4ebb-a332-b0a0bf2dfb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:41.047376 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa114e9-abd9-4ebb-a332-b0a0bf2dfb25 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:41.050837 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa114e9-abd9-4ebb-a332-b0a0bf2dfb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:41.051643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa114e9-abd9-4ebb-a332-b0a0bf2dfb25 admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:41.052402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa114e9-abd9-4ebb-a332-b0a0bf2dfb25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:41.063641 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa114e9-abd9-4ebb-a332-b0a0bf2dfb25 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:41.142869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325ffb5-a2ec-4707-9962-fdd250f843aa admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:41.155021 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 847/1698] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:45:40 2022] POST /identity/v3/users => generated 283 bytes in 602 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:41.155951 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa114e9-abd9-4ebb-a332-b0a0bf2dfb25 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:41.166438 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266d772-bf32-46fe-87fe-990dd3feb7c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:41.168622 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266d772-bf32-46fe-87fe-990dd3feb7c6 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:41.169522 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266d772-bf32-46fe-87fe-990dd3feb7c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:41.169864 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266d772-bf32-46fe-87fe-990dd3feb7c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:41.203213 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7266d772-bf32-46fe-87fe-990dd3feb7c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:41.206556 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa114e9-abd9-4ebb-a332-b0a0bf2dfb25 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:41.207522 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7266d772-bf32-46fe-87fe-990dd3feb7c6 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:41.220323 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee2a0c-a04e-4fa2-b2f2-60caaf596986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:41.221320 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee2a0c-a04e-4fa2-b2f2-60caaf596986 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:41.222420 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee2a0c-a04e-4fa2-b2f2-60caaf596986 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:41.222700 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee2a0c-a04e-4fa2-b2f2-60caaf596986 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:41.223471 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ee2a0c-a04e-4fa2-b2f2-60caaf596986 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:41.238465 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ee2a0c-a04e-4fa2-b2f2-60caaf596986 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:41.243835 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 852/1699] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:41.257610 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266d772-bf32-46fe-87fe-990dd3feb7c6 service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:41.310784 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 848/1700] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:41 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:41.316939 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee2a0c-a04e-4fa2-b2f2-60caaf596986 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:41.322541 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266d772-bf32-46fe-87fe-990dd3feb7c6 service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:41.335066 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c10c76a-1b7a-46d4-94d0-428f641f16e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:41.337786 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10c76a-1b7a-46d4-94d0-428f641f16e3 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:41.339063 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10c76a-1b7a-46d4-94d0-428f641f16e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:41.339672 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10c76a-1b7a-46d4-94d0-428f641f16e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:41.340819 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c10c76a-1b7a-46d4-94d0-428f641f16e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:41.345300 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c10c76a-1b7a-46d4-94d0-428f641f16e3 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:41.347078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee2a0c-a04e-4fa2-b2f2-60caaf596986 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:41.360358 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc1b7c-61f9-46be-82ff-4f3368a2e89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:41.360816 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc1b7c-61f9-46be-82ff-4f3368a2e89f admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:41.361133 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc1b7c-61f9-46be-82ff-4f3368a2e89f admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:41.361451 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc1b7c-61f9-46be-82ff-4f3368a2e89f admin admin] PATH_INFO: `/v3/projects/ecb3548713e744c69a4729856f837677/users/9fb4cbf28159468d848fca787d27533d/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:41.386597 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 853/1701] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:41 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:41.387612 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c10c76a-1b7a-46d4-94d0-428f641f16e3 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:41.404766 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c10c76a-1b7a-46d4-94d0-428f641f16e3 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:41.408098 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc1b7c-61f9-46be-82ff-4f3368a2e89f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecb3548713e744c69a4729856f837677, user_id=9fb4cbf28159468d848fca787d27533d, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:41.414340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc1b7c-61f9-46be-82ff-4f3368a2e89f admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:41.425750 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ddba5-28eb-47d1-a8f4-1a6f8ab1cd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:41.426702 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ddba5-28eb-47d1-a8f4-1a6f8ab1cd4a admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:41.427541 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ddba5-28eb-47d1-a8f4-1a6f8ab1cd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:41.428295 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ddba5-28eb-47d1-a8f4-1a6f8ab1cd4a admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:41.431246 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ddba5-28eb-47d1-a8f4-1a6f8ab1cd4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:41.433036 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ddba5-28eb-47d1-a8f4-1a6f8ab1cd4a admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:41.448744 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:41.457484 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 849/1702] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:45:41 2022] PUT /identity/v3/projects/ecb3548713e744c69a4729856f837677/users/9fb4cbf28159468d848fca787d27533d/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:41.461717 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 854/1703] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:41 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:41.464346 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ddba5-28eb-47d1-a8f4-1a6f8ab1cd4a admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:41.464482 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc1b7c-61f9-46be-82ff-4f3368a2e89f admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:41.495833 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ddba5-28eb-47d1-a8f4-1a6f8ab1cd4a admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:41.597249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:41.693738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc1b7c-61f9-46be-82ff-4f3368a2e89f admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:41.716780 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6977d4d6-7245-470f-9d47-00333c0605e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:41.717483 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977d4d6-7245-470f-9d47-00333c0605e1 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:41.718272 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977d4d6-7245-470f-9d47-00333c0605e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:41.718656 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977d4d6-7245-470f-9d47-00333c0605e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:41.719695 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6977d4d6-7245-470f-9d47-00333c0605e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:41.721098 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847155ca-c5e5-4fec-8d6e-cc83c2b19ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:41.722059 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6977d4d6-7245-470f-9d47-00333c0605e1 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:41.722269 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847155ca-c5e5-4fec-8d6e-cc83c2b19ebb admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:41.722567 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847155ca-c5e5-4fec-8d6e-cc83c2b19ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:41.722850 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847155ca-c5e5-4fec-8d6e-cc83c2b19ebb admin admin] PATH_INFO: `/v3/projects/fc11b5c5b8d54b478df89e08aba0ad86/users/9e307a7fb4394ff3812336fb8cc0e4ab/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:41.746753 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 850/1704] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:41 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:41.749723 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6977d4d6-7245-470f-9d47-00333c0605e1 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:41.779882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847155ca-c5e5-4fec-8d6e-cc83c2b19ebb admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc11b5c5b8d54b478df89e08aba0ad86, user_id=9e307a7fb4394ff3812336fb8cc0e4ab, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:41.787738 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847155ca-c5e5-4fec-8d6e-cc83c2b19ebb admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:41.804079 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6977d4d6-7245-470f-9d47-00333c0605e1 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:41.830507 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 855/1705] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:45:41 2022] PUT /identity/v3/projects/fc11b5c5b8d54b478df89e08aba0ad86/users/9e307a7fb4394ff3812336fb8cc0e4ab/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:41.833947 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918da22-803d-4d27-8ccc-a913a8497c90 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:41.834304 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918da22-803d-4d27-8ccc-a913a8497c90 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:41.834590 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918da22-803d-4d27-8ccc-a913a8497c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:41.910271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-5918da22-803d-4d27-8ccc-a913a8497c90 None None] MFA Rules not processed for user `9e307a7fb4394ff3812336fb8cc0e4ab`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:41.911401 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-5918da22-803d-4d27-8ccc-a913a8497c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:41.957186 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629dc2b0-6b2d-455d-924d-5772f20a66a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:41.958110 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629dc2b0-6b2d-455d-924d-5772f20a66a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:41.958394 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629dc2b0-6b2d-455d-924d-5772f20a66a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:41.958671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629dc2b0-6b2d-455d-924d-5772f20a66a8 admin admin] PATH_INFO: `/v3/projects/ecb3548713e744c69a4729856f837677/users/9fb4cbf28159468d848fca787d27533d/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:41.967451 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629dc2b0-6b2d-455d-924d-5772f20a66a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecb3548713e744c69a4729856f837677, user_id=9fb4cbf28159468d848fca787d27533d, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:41.974697 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629dc2b0-6b2d-455d-924d-5772f20a66a8 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:42.033255 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 851/1706] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:45:41 2022] PUT /identity/v3/projects/ecb3548713e744c69a4729856f837677/users/9fb4cbf28159468d848fca787d27533d/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:42.037955 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495453c-d6f2-400e-b7ab-b5396302a950 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:42.041202 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495453c-d6f2-400e-b7ab-b5396302a950 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:42.043812 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495453c-d6f2-400e-b7ab-b5396302a950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:42.097875 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-5495453c-d6f2-400e-b7ab-b5396302a950 None None] MFA Rules not processed for user `b9f250f5dbc54d2e9293962ece848f6c`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:42.100618 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-5495453c-d6f2-400e-b7ab-b5396302a950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:42.118826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 856/1707] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:41 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:42.118826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f206c9-65de-4527-9fed-4cc8e6610db9 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:42.119018 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f206c9-65de-4527-9fed-4cc8e6610db9 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:42.120111 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f206c9-65de-4527-9fed-4cc8e6610db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:42.196766 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-39f206c9-65de-4527-9fed-4cc8e6610db9 None None] MFA Rules not processed for user `9fb4cbf28159468d848fca787d27533d`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:42.198100 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-39f206c9-65de-4527-9fed-4cc8e6610db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:42.391723 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 852/1708] 10.222.0.39 () {58 vars in 1015 bytes} [Wed Jul 6 13:45:42 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:42.395791 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495453c-d6f2-400e-b7ab-b5396302a950 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:42.478850 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:42.552513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495453c-d6f2-400e-b7ab-b5396302a950 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:42.579170 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb37ea-36fc-41ed-b8df-3c9b75b54faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:42.579170 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb37ea-36fc-41ed-b8df-3c9b75b54faa admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:42.579170 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb37ea-36fc-41ed-b8df-3c9b75b54faa admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:42.579170 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb37ea-36fc-41ed-b8df-3c9b75b54faa admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:42.579601 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb37ea-36fc-41ed-b8df-3c9b75b54faa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:42.583251 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb37ea-36fc-41ed-b8df-3c9b75b54faa admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:42.597475 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 857/1709] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:42 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 482 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:42.598586 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f206c9-65de-4527-9fed-4cc8e6610db9 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:42.632105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f206c9-65de-4527-9fed-4cc8e6610db9 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:42.655037 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382fc41f-a487-483e-8b29-6ae74d23b750 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:42.655869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382fc41f-a487-483e-8b29-6ae74d23b750 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:42.656163 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382fc41f-a487-483e-8b29-6ae74d23b750 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:42.656551 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382fc41f-a487-483e-8b29-6ae74d23b750 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:42.683718 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 853/1710] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:45:42 2022] POST /identity/v3/projects => generated 365 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:42.684457 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb37ea-36fc-41ed-b8df-3c9b75b54faa admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:42.687960 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-9afb37ea-36fc-41ed-b8df-3c9b75b54faa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:42.689541 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382fc41f-a487-483e-8b29-6ae74d23b750 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:42.691901 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382fc41f-a487-483e-8b29-6ae74d23b750 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:42.754379 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 858/1711] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:42.759748 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382fc41f-a487-483e-8b29-6ae74d23b750 service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:42.834364 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382fc41f-a487-483e-8b29-6ae74d23b750 service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:42.887111 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:43.038189 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb37ea-36fc-41ed-b8df-3c9b75b54faa admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:43.058834 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d9aec-3700-47d0-ad54-a69733fe5059 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:43.065554 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d9aec-3700-47d0-ad54-a69733fe5059 service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:43.066628 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d9aec-3700-47d0-ad54-a69733fe5059 service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:43.067108 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d9aec-3700-47d0-ad54-a69733fe5059 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:43.164957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d726c3-2eb2-4fb5-8b23-0db524ede577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:43.169250 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d726c3-2eb2-4fb5-8b23-0db524ede577 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:43.169588 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d726c3-2eb2-4fb5-8b23-0db524ede577 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:43.169966 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d726c3-2eb2-4fb5-8b23-0db524ede577 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:43.170725 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d726c3-2eb2-4fb5-8b23-0db524ede577 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:43.181726 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d726c3-2eb2-4fb5-8b23-0db524ede577 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:43.230117 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 859/1712] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:45:42 2022] POST /identity/v3/users => generated 289 bytes in 471 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:43.231576 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d726c3-2eb2-4fb5-8b23-0db524ede577 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:43.237769 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-b7d726c3-2eb2-4fb5-8b23-0db524ede577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:43.457788 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d9aec-3700-47d0-ad54-a69733fe5059 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:43.466104 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d726c3-2eb2-4fb5-8b23-0db524ede577 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:43.466454 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d9aec-3700-47d0-ad54-a69733fe5059 service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:43.471125 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d0494-0165-4817-86c5-b6cb72d9e33e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:43.471810 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d0494-0165-4817-86c5-b6cb72d9e33e service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:43.472101 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d0494-0165-4817-86c5-b6cb72d9e33e service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:43.473496 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d0494-0165-4817-86c5-b6cb72d9e33e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:43.484206 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d0494-0165-4817-86c5-b6cb72d9e33e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:43.485865 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d0494-0165-4817-86c5-b6cb72d9e33e service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:43.504722 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 854/1713] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:42 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 822 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:43.524555 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d9aec-3700-47d0-ad54-a69733fe5059 service cinder] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:43.530701 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 860/1714] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:43.608462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d9aec-3700-47d0-ad54-a69733fe5059 service cinder] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:43.617883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf559a61-38f2-4e7e-9a4b-7a004790d24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:43.618875 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf559a61-38f2-4e7e-9a4b-7a004790d24f admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:43.619946 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf559a61-38f2-4e7e-9a4b-7a004790d24f admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:43.620421 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf559a61-38f2-4e7e-9a4b-7a004790d24f admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:43.623163 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf559a61-38f2-4e7e-9a4b-7a004790d24f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:43.626059 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf559a61-38f2-4e7e-9a4b-7a004790d24f admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:43.681298 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d0494-0165-4817-86c5-b6cb72d9e33e service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:43.683032 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 855/1715] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:43.683258 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-a58d0494-0165-4817-86c5-b6cb72d9e33e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:43.692285 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf559a61-38f2-4e7e-9a4b-7a004790d24f admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:43.738607 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d0494-0165-4817-86c5-b6cb72d9e33e service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:43.749080 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf559a61-38f2-4e7e-9a4b-7a004790d24f admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:43.759070 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71526ce-1cf5-49a5-a65b-1b8f3d6c77a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:43.759825 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71526ce-1cf5-49a5-a65b-1b8f3d6c77a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:43.760114 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71526ce-1cf5-49a5-a65b-1b8f3d6c77a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:43.760390 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71526ce-1cf5-49a5-a65b-1b8f3d6c77a3 admin admin] PATH_INFO: `/v3/users/0147bf710e1a42b79668bbd6936c4e60` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:43.766329 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71526ce-1cf5-49a5-a65b-1b8f3d6c77a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0147bf710e1a42b79668bbd6936c4e60)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:43.776738 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71526ce-1cf5-49a5-a65b-1b8f3d6c77a3 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:43.807751 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3807857c-555c-47bf-a20b-d20e891266ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:43.817342 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807857c-555c-47bf-a20b-d20e891266ac admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:43.817901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807857c-555c-47bf-a20b-d20e891266ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:43.820047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807857c-555c-47bf-a20b-d20e891266ac admin admin] PATH_INFO: `/v3/projects/2a56989b6c864610bceb0192c52d8cef/users/eb88a84552d14bfca6627a38d7c2d385/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:43.958827 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3807857c-555c-47bf-a20b-d20e891266ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a56989b6c864610bceb0192c52d8cef, user_id=eb88a84552d14bfca6627a38d7c2d385, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:43.977845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3807857c-555c-47bf-a20b-d20e891266ac admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:44.032618 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b71526ce-1cf5-49a5-a65b-1b8f3d6c77a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0147bf710e1a42b79668bbd6936c4e60'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:44.050851 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 856/1716] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:45:43 2022] PUT /identity/v3/projects/2a56989b6c864610bceb0192c52d8cef/users/eb88a84552d14bfca6627a38d7c2d385/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:44.061630 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3807857c-555c-47bf-a20b-d20e891266ac admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:44.077346 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b71526ce-1cf5-49a5-a65b-1b8f3d6c77a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0147bf710e1a42b79668bbd6936c4e60'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:44.134680 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b71526ce-1cf5-49a5-a65b-1b8f3d6c77a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0147bf710e1a42b79668bbd6936c4e60'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:44.135996 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b71526ce-1cf5-49a5-a65b-1b8f3d6c77a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0147bf710e1a42b79668bbd6936c4e60'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:44.173501 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b71526ce-1cf5-49a5-a65b-1b8f3d6c77a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0147bf710e1a42b79668bbd6936c4e60'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:44.190308 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 861/1717] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:45:43 2022] DELETE /identity/v3/users/0147bf710e1a42b79668bbd6936c4e60 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:45:44.195643 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90eeb67-3d45-422a-b7dd-37e697cb8e05 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:44.196958 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90eeb67-3d45-422a-b7dd-37e697cb8e05 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:44.197891 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90eeb67-3d45-422a-b7dd-37e697cb8e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:44.248037 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:44.271779 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-b90eeb67-3d45-422a-b7dd-37e697cb8e05 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:44.280035 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-b90eeb67-3d45-422a-b7dd-37e697cb8e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:44.384276 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3807857c-555c-47bf-a20b-d20e891266ac admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:44.397178 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:44.625088 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec2768-c00b-461b-874c-b47f1735dfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:44.625806 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec2768-c00b-461b-874c-b47f1735dfe3 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:44.626077 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec2768-c00b-461b-874c-b47f1735dfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:44.626354 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec2768-c00b-461b-874c-b47f1735dfe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:44.627262 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec2768-c00b-461b-874c-b47f1735dfe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:44.628914 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec2768-c00b-461b-874c-b47f1735dfe3 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:44.636192 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 862/1718] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 439 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:44.639462 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90eeb67-3d45-422a-b7dd-37e697cb8e05 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:44.641492 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-b90eeb67-3d45-422a-b7dd-37e697cb8e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:44.663894 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90eeb67-3d45-422a-b7dd-37e697cb8e05 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:44.686336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 857/1719] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 635 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:44.688170 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac1139-7485-46e1-8285-63898427865c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:44.688707 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac1139-7485-46e1-8285-63898427865c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:44.689020 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac1139-7485-46e1-8285-63898427865c admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:44.689440 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac1139-7485-46e1-8285-63898427865c admin admin] PATH_INFO: `/v3/users/1b405df0afb247808aea6737cadc14c7` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:44.691162 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec2768-c00b-461b-874c-b47f1735dfe3 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:44.692801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ac1139-7485-46e1-8285-63898427865c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b405df0afb247808aea6737cadc14c7)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:44.694459 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ac1139-7485-46e1-8285-63898427865c admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:44.701907 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-fcec2768-c00b-461b-874c-b47f1735dfe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:44.755668 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec2768-c00b-461b-874c-b47f1735dfe3 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:44.769826 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3662a7e-386c-4dd7-a87b-2886b9cc2198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:44.770993 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3662a7e-386c-4dd7-a87b-2886b9cc2198 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:44.771443 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3662a7e-386c-4dd7-a87b-2886b9cc2198 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:44.771949 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3662a7e-386c-4dd7-a87b-2886b9cc2198 admin admin] PATH_INFO: `/v3/projects/2a56989b6c864610bceb0192c52d8cef/users/eb88a84552d14bfca6627a38d7c2d385/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:44.779503 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3662a7e-386c-4dd7-a87b-2886b9cc2198 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a56989b6c864610bceb0192c52d8cef, user_id=eb88a84552d14bfca6627a38d7c2d385, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:44.782619 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3662a7e-386c-4dd7-a87b-2886b9cc2198 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:44.815184 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 858/1720] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:45:44 2022] PUT /identity/v3/projects/2a56989b6c864610bceb0192c52d8cef/users/eb88a84552d14bfca6627a38d7c2d385/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:44.816761 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3662a7e-386c-4dd7-a87b-2886b9cc2198 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:44.821617 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-e3662a7e-386c-4dd7-a87b-2886b9cc2198 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:44.825162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-55ac1139-7485-46e1-8285-63898427865c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b405df0afb247808aea6737cadc14c7'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:44.838136 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-55ac1139-7485-46e1-8285-63898427865c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b405df0afb247808aea6737cadc14c7'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:44.854235 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-55ac1139-7485-46e1-8285-63898427865c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b405df0afb247808aea6737cadc14c7'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:44.856277 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-55ac1139-7485-46e1-8285-63898427865c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b405df0afb247808aea6737cadc14c7'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:44.907183 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-55ac1139-7485-46e1-8285-63898427865c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b405df0afb247808aea6737cadc14c7'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:44.923315 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 863/1721] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:45:44 2022] DELETE /identity/v3/users/1b405df0afb247808aea6737cadc14c7 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:44.924873 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd49da-b48a-4c10-b3a1-f6d6c13b8a10 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:44.928022 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd49da-b48a-4c10-b3a1-f6d6c13b8a10 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:44.928535 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd49da-b48a-4c10-b3a1-f6d6c13b8a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:45.006021 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3662a7e-386c-4dd7-a87b-2886b9cc2198 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:45.010577 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-e1bd49da-b48a-4c10-b3a1-f6d6c13b8a10 None None] MFA Rules not processed for user `eb88a84552d14bfca6627a38d7c2d385`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:45.012436 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-e1bd49da-b48a-4c10-b3a1-f6d6c13b8a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:45.176566 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b363dbd-3b44-480e-a0eb-145ac6ffa9da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:45.177605 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b363dbd-3b44-480e-a0eb-145ac6ffa9da service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:45.178014 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b363dbd-3b44-480e-a0eb-145ac6ffa9da service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:45.179829 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b363dbd-3b44-480e-a0eb-145ac6ffa9da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:45.184430 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-2b363dbd-3b44-480e-a0eb-145ac6ffa9da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:45.251765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 864/1722] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:44 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:45.254578 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae35f99-2ca1-45d1-9391-20d7ab098827 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:45.256922 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae35f99-2ca1-45d1-9391-20d7ab098827 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:45.257376 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae35f99-2ca1-45d1-9391-20d7ab098827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:45.318889 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:45.334745 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-7ae35f99-2ca1-45d1-9391-20d7ab098827 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:45.341356 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-7ae35f99-2ca1-45d1-9391-20d7ab098827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:45.403412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:45.423549 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b363dbd-3b44-480e-a0eb-145ac6ffa9da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:45.430065 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b363dbd-3b44-480e-a0eb-145ac6ffa9da service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:45.498384 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 859/1723] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 682 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:45.500612 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b363dbd-3b44-480e-a0eb-145ac6ffa9da service neutron] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:45.511757 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-2b363dbd-3b44-480e-a0eb-145ac6ffa9da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:45.654146 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 865/1724] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:45.821072 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae35f99-2ca1-45d1-9391-20d7ab098827 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:45.823184 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-7ae35f99-2ca1-45d1-9391-20d7ab098827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:45.873706 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae35f99-2ca1-45d1-9391-20d7ab098827 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:45.895923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de218740-6808-458d-82bc-954888cd50d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:45.902669 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de218740-6808-458d-82bc-954888cd50d8 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:45.905440 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de218740-6808-458d-82bc-954888cd50d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:45.906030 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de218740-6808-458d-82bc-954888cd50d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:45.940554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de218740-6808-458d-82bc-954888cd50d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:45.948384 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de218740-6808-458d-82bc-954888cd50d8 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:45.955638 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b363dbd-3b44-480e-a0eb-145ac6ffa9da service neutron] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:45.972677 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b074aea5-d142-4d4d-aa52-85970a5c51f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:45.973295 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074aea5-d142-4d4d-aa52-85970a5c51f4 service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:45.973606 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074aea5-d142-4d4d-aa52-85970a5c51f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:45.973995 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074aea5-d142-4d4d-aa52-85970a5c51f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:45.990315 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 866/1725] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 13:45:45.996762 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b074aea5-d142-4d4d-aa52-85970a5c51f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:45.998522 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b074aea5-d142-4d4d-aa52-85970a5c51f4 service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:46.026736 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 860/1726] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:45 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:46.393727 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de218740-6808-458d-82bc-954888cd50d8 service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:46.409975 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de218740-6808-458d-82bc-954888cd50d8 service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:46.414026 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a74c96-afe1-4901-b430-cad46667e7fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:46.415028 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a74c96-afe1-4901-b430-cad46667e7fe service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:46.415309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a74c96-afe1-4901-b430-cad46667e7fe service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:46.415835 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a74c96-afe1-4901-b430-cad46667e7fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:46.419503 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-f3a74c96-afe1-4901-b430-cad46667e7fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:46.623331 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a74c96-afe1-4901-b430-cad46667e7fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:46.624816 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a74c96-afe1-4901-b430-cad46667e7fe service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:46.667153 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 867/1727] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:46 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:47.490869 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6babb1d3-85be-419b-814a-498592a21bb5 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:47.491460 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6babb1d3-85be-419b-814a-498592a21bb5 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:47.491460 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6babb1d3-85be-419b-814a-498592a21bb5 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:47.493146 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 861/1728] 10.222.0.20 () {54 vars in 921 bytes} [Wed Jul 6 13:45:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:45:47.560216 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5802e40-c7d5-46e2-a6e1-704451f6d7c8 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:47.560975 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5802e40-c7d5-46e2-a6e1-704451f6d7c8 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:47.560975 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5802e40-c7d5-46e2-a6e1-704451f6d7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:47.609059 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-f5802e40-c7d5-46e2-a6e1-704451f6d7c8 None None] MFA Rules not processed for user `6647eb1e8a1544ef9d3a9806f2d1fc51`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:47.610303 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-f5802e40-c7d5-46e2-a6e1-704451f6d7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:47.908724 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 868/1729] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Jul 6 13:45:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:48.338718 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6babb1d3-85be-419b-814a-498592a21bb5 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:48.340489 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-6babb1d3-85be-419b-814a-498592a21bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:48.380042 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6babb1d3-85be-419b-814a-498592a21bb5 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:48.412608 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9150a1-5a60-412a-b2e1-28b16ec587fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:48.413529 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9150a1-5a60-412a-b2e1-28b16ec587fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:48.413815 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9150a1-5a60-412a-b2e1-28b16ec587fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:48.414632 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9150a1-5a60-412a-b2e1-28b16ec587fa admin admin] PATH_INFO: `/v3/projects/7c55552b8bb945818f81c7ed751bf6cf` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:48.419269 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9150a1-5a60-412a-b2e1-28b16ec587fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c55552b8bb945818f81c7ed751bf6cf)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:48.431967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9150a1-5a60-412a-b2e1-28b16ec587fa admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:48.570699 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-3a9150a1-5a60-412a-b2e1-28b16ec587fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c55552b8bb945818f81c7ed751bf6cf', 'request_id': 'req-3a9150a1-5a60-412a-b2e1-28b16ec587fa'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:48.626363 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-3a9150a1-5a60-412a-b2e1-28b16ec587fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c55552b8bb945818f81c7ed751bf6cf', 'request_id': 'req-3a9150a1-5a60-412a-b2e1-28b16ec587fa'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:48.652946 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-3a9150a1-5a60-412a-b2e1-28b16ec587fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c55552b8bb945818f81c7ed751bf6cf', 'request_id': 'req-3a9150a1-5a60-412a-b2e1-28b16ec587fa'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:48.715247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-3a9150a1-5a60-412a-b2e1-28b16ec587fa admin admin] The token cache is being invalidate because project 7c55552b8bb945818f81c7ed751bf6cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:45:48.716034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-3a9150a1-5a60-412a-b2e1-28b16ec587fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:48.719210 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-3a9150a1-5a60-412a-b2e1-28b16ec587fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:48.726295 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 862/1730] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:45:48 2022] DELETE /identity/v3/projects/7c55552b8bb945818f81c7ed751bf6cf => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:45:48.732132 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905f6e0-cc61-4bc0-ab1e-be77879a841c None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:48.732575 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905f6e0-cc61-4bc0-ab1e-be77879a841c None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:48.732805 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905f6e0-cc61-4bc0-ab1e-be77879a841c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:48.815749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66557e99-232d-49ae-9369-b87a05c56289 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:48.816035 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66557e99-232d-49ae-9369-b87a05c56289 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:48.816485 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66557e99-232d-49ae-9369-b87a05c56289 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:48.818099 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 863/1731] 10.222.0.20 () {54 vars in 921 bytes} [Wed Jul 6 13:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:45:48.823594 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-3905f6e0-cc61-4bc0-ab1e-be77879a841c None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:48.832818 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-3905f6e0-cc61-4bc0-ab1e-be77879a841c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:48.960242 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:49.045652 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 869/1732] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:49.049332 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c472f3-844a-4619-a9d0-110c1f50e54f None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:49.049925 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c472f3-844a-4619-a9d0-110c1f50e54f None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:49.050634 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c472f3-844a-4619-a9d0-110c1f50e54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:49.093997 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-65c472f3-844a-4619-a9d0-110c1f50e54f None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:49.095995 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-65c472f3-844a-4619-a9d0-110c1f50e54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:49.120187 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 864/1733] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:49.121772 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905f6e0-cc61-4bc0-ab1e-be77879a841c None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:49.139307 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905f6e0-cc61-4bc0-ab1e-be77879a841c None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:49.151875 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea28cac-fa37-469b-9b70-21c7ed8a15a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:49.152647 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea28cac-fa37-469b-9b70-21c7ed8a15a6 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:49.152846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea28cac-fa37-469b-9b70-21c7ed8a15a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:49.153162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea28cac-fa37-469b-9b70-21c7ed8a15a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:49.153914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea28cac-fa37-469b-9b70-21c7ed8a15a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:45:49.154229 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea28cac-fa37-469b-9b70-21c7ed8a15a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:49.155961 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea28cac-fa37-469b-9b70-21c7ed8a15a6 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:49.216655 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c472f3-844a-4619-a9d0-110c1f50e54f None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:49.237836 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c472f3-844a-4619-a9d0-110c1f50e54f None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:49.238482 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 870/1734] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:45:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:45:49.257979 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ddcdcc-d7af-4fe6-b6fd-295652a4d92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:49.261300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ddcdcc-d7af-4fe6-b6fd-295652a4d92a admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:49.265281 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ddcdcc-d7af-4fe6-b6fd-295652a4d92a admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:49.265801 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ddcdcc-d7af-4fe6-b6fd-295652a4d92a admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:49.267730 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ddcdcc-d7af-4fe6-b6fd-295652a4d92a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:49.274364 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ddcdcc-d7af-4fe6-b6fd-295652a4d92a admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:49.361063 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 865/1735] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:45:49 2022] POST /identity/v3/projects => generated 403 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:49.365233 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea28cac-fa37-469b-9b70-21c7ed8a15a6 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:49.384914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea28cac-fa37-469b-9b70-21c7ed8a15a6 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:49.450599 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42fadb-f3a9-4cab-82a0-d16238844952 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:49.451161 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42fadb-f3a9-4cab-82a0-d16238844952 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:49.451733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42fadb-f3a9-4cab-82a0-d16238844952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:49.556761 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-cc42fadb-f3a9-4cab-82a0-d16238844952 None None] MFA Rules not processed for user `6647eb1e8a1544ef9d3a9806f2d1fc51`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:49.559603 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-cc42fadb-f3a9-4cab-82a0-d16238844952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:49.622896 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac4e07-34cc-40c1-a02b-e9f083abdf44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:49.623631 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac4e07-34cc-40c1-a02b-e9f083abdf44 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:49.623913 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac4e07-34cc-40c1-a02b-e9f083abdf44 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:49.624343 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac4e07-34cc-40c1-a02b-e9f083abdf44 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:49.625197 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ac4e07-34cc-40c1-a02b-e9f083abdf44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:49.627246 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ac4e07-34cc-40c1-a02b-e9f083abdf44 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:49.657464 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:49.697089 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 871/1736] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:45:49 2022] POST /identity/v3/users => generated 302 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:49.698052 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac4e07-34cc-40c1-a02b-e9f083abdf44 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:49.702774 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-a0ac4e07-34cc-40c1-a02b-e9f083abdf44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:49.784785 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 866/1737] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Jul 6 13:45:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:49.786283 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc42fadb-f3a9-4cab-82a0-d16238844952 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:49.814666 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc42fadb-f3a9-4cab-82a0-d16238844952 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:49.826697 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b0832-67a8-4584-b1c7-e86c8f4cd50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:49.830775 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b0832-67a8-4584-b1c7-e86c8f4cd50f admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:49.831072 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b0832-67a8-4584-b1c7-e86c8f4cd50f admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:49.832590 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b0832-67a8-4584-b1c7-e86c8f4cd50f admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:49.835552 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b0832-67a8-4584-b1c7-e86c8f4cd50f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:49.840516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b0832-67a8-4584-b1c7-e86c8f4cd50f admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:49.894680 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 867/1738] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:49.901100 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b0832-67a8-4584-b1c7-e86c8f4cd50f admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:49.923542 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b0832-67a8-4584-b1c7-e86c8f4cd50f admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:49.939138 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac4e07-34cc-40c1-a02b-e9f083abdf44 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:49.946660 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3749b2b-41cf-40cb-a32f-6b1703418786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:49.947435 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3749b2b-41cf-40cb-a32f-6b1703418786 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:49.947785 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3749b2b-41cf-40cb-a32f-6b1703418786 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:49.948385 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3749b2b-41cf-40cb-a32f-6b1703418786 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:49.949229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3749b2b-41cf-40cb-a32f-6b1703418786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:49.951843 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3749b2b-41cf-40cb-a32f-6b1703418786 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:49.966476 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55341604-5213-44bd-b26f-47642d2a158c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:49.967291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55341604-5213-44bd-b26f-47642d2a158c service nova] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:49.967600 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55341604-5213-44bd-b26f-47642d2a158c service nova] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:49.967969 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55341604-5213-44bd-b26f-47642d2a158c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:49.970693 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-55341604-5213-44bd-b26f-47642d2a158c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:49.991913 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 868/1739] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:49.995006 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3749b2b-41cf-40cb-a32f-6b1703418786 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:50.014001 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55341604-5213-44bd-b26f-47642d2a158c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:50.015268 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-b3749b2b-41cf-40cb-a32f-6b1703418786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:50.015831 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55341604-5213-44bd-b26f-47642d2a158c service nova] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:50.029033 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 872/1740] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:50.029963 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55341604-5213-44bd-b26f-47642d2a158c service nova] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:50.032242 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3749b2b-41cf-40cb-a32f-6b1703418786 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:50.040040 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412926be-6003-4588-9f42-431d0e4258d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:50.041217 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412926be-6003-4588-9f42-431d0e4258d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:50.041604 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412926be-6003-4588-9f42-431d0e4258d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:50.041739 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55341604-5213-44bd-b26f-47642d2a158c service nova] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:50.042059 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412926be-6003-4588-9f42-431d0e4258d6 admin admin] PATH_INFO: `/v3/projects/fb70788b58f648acbdfe25cc67c11ace` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:50.049303 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b2d82-b3a1-4b9e-becb-cd0e4acda4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:50.050225 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b2d82-b3a1-4b9e-becb-cd0e4acda4ed admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:50.050516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b2d82-b3a1-4b9e-becb-cd0e4acda4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:50.050801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b2d82-b3a1-4b9e-becb-cd0e4acda4ed admin admin] PATH_INFO: `/v3/projects/40879f94775e43c982ff1a19c82db6fb/users/2d4da8cc84fe4716a316183a5a88096b/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:50.069886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412926be-6003-4588-9f42-431d0e4258d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb70788b58f648acbdfe25cc67c11ace)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:50.077188 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412926be-6003-4588-9f42-431d0e4258d6 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:50.095507 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b2d82-b3a1-4b9e-becb-cd0e4acda4ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=40879f94775e43c982ff1a19c82db6fb, user_id=2d4da8cc84fe4716a316183a5a88096b, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:50.102292 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b2d82-b3a1-4b9e-becb-cd0e4acda4ed admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:50.135926 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 873/1741] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:45:50 2022] PUT /identity/v3/projects/40879f94775e43c982ff1a19c82db6fb/users/2d4da8cc84fe4716a316183a5a88096b/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:50.137645 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7926ec-13d0-4573-9e28-65d43110bf5c None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:50.137954 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7926ec-13d0-4573-9e28-65d43110bf5c None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:50.138279 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7926ec-13d0-4573-9e28-65d43110bf5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:50.187696 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-3f7926ec-13d0-4573-9e28-65d43110bf5c None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:50.191500 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-3f7926ec-13d0-4573-9e28-65d43110bf5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:50.206740 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:50.240167 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-412926be-6003-4588-9f42-431d0e4258d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb70788b58f648acbdfe25cc67c11ace', 'request_id': 'req-412926be-6003-4588-9f42-431d0e4258d6'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:50.275770 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-412926be-6003-4588-9f42-431d0e4258d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb70788b58f648acbdfe25cc67c11ace', 'request_id': 'req-412926be-6003-4588-9f42-431d0e4258d6'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:50.307496 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-412926be-6003-4588-9f42-431d0e4258d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb70788b58f648acbdfe25cc67c11ace', 'request_id': 'req-412926be-6003-4588-9f42-431d0e4258d6'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:50.340836 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-412926be-6003-4588-9f42-431d0e4258d6 admin admin] The token cache is being invalidate because project fb70788b58f648acbdfe25cc67c11ace was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:45:50.341221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-412926be-6003-4588-9f42-431d0e4258d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:50.342178 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-412926be-6003-4588-9f42-431d0e4258d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:50.349122 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 869/1742] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:45:49 2022] DELETE /identity/v3/projects/fb70788b58f648acbdfe25cc67c11ace => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:50.350188 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412926be-6003-4588-9f42-431d0e4258d6 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:50.353136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-412926be-6003-4588-9f42-431d0e4258d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:50.390602 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 874/1743] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:45:50.391586 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7926ec-13d0-4573-9e28-65d43110bf5c None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:50.396829 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-3f7926ec-13d0-4573-9e28-65d43110bf5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:50.422891 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7926ec-13d0-4573-9e28-65d43110bf5c None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:50.439894 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e80a5-2fee-4c17-a941-7437d7b8c178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:50.441178 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e80a5-2fee-4c17-a941-7437d7b8c178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:50.441477 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e80a5-2fee-4c17-a941-7437d7b8c178 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:50.441785 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e80a5-2fee-4c17-a941-7437d7b8c178 admin admin] PATH_INFO: `/v3/users/9fb4cbf28159468d848fca787d27533d` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:50.444317 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3e80a5-2fee-4c17-a941-7437d7b8c178 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fb4cbf28159468d848fca787d27533d)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:50.451349 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3e80a5-2fee-4c17-a941-7437d7b8c178 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:50.565801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-bc3e80a5-2fee-4c17-a941-7437d7b8c178 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fb4cbf28159468d848fca787d27533d'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:50.574803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412926be-6003-4588-9f42-431d0e4258d6 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:50.581314 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-bc3e80a5-2fee-4c17-a941-7437d7b8c178 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fb4cbf28159468d848fca787d27533d'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:50.596305 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-bc3e80a5-2fee-4c17-a941-7437d7b8c178 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fb4cbf28159468d848fca787d27533d'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:50.598520 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-bc3e80a5-2fee-4c17-a941-7437d7b8c178 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fb4cbf28159468d848fca787d27533d'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:50.601113 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1bfc1-ad4f-4453-a6e2-c8d27fe22617 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:50.612653 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1bfc1-ad4f-4453-a6e2-c8d27fe22617 service nova] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:50.613039 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1bfc1-ad4f-4453-a6e2-c8d27fe22617 service nova] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:50.615256 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1bfc1-ad4f-4453-a6e2-c8d27fe22617 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:50.625524 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-e9a1bfc1-ad4f-4453-a6e2-c8d27fe22617 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:50.632117 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-bc3e80a5-2fee-4c17-a941-7437d7b8c178 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fb4cbf28159468d848fca787d27533d'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:50.647811 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 875/1744] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:45:50 2022] DELETE /identity/v3/users/9fb4cbf28159468d848fca787d27533d => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:50.654805 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e80a5-2fee-4c17-a941-7437d7b8c178 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:50.659107 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-bc3e80a5-2fee-4c17-a941-7437d7b8c178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:50.682151 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a1bfc1-ad4f-4453-a6e2-c8d27fe22617 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:50.690763 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a1bfc1-ad4f-4453-a6e2-c8d27fe22617 service nova] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:50.733430 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 870/1745] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:50.804718 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1bfc1-ad4f-4453-a6e2-c8d27fe22617 service nova] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:50.806871 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-e9a1bfc1-ad4f-4453-a6e2-c8d27fe22617 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:50.818982 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e80a5-2fee-4c17-a941-7437d7b8c178 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:50.825083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1330660f-22c5-4002-95bb-e1393e75642e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:50.826923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330660f-22c5-4002-95bb-e1393e75642e service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:50.827436 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330660f-22c5-4002-95bb-e1393e75642e service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:50.828153 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330660f-22c5-4002-95bb-e1393e75642e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:50.832185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-1330660f-22c5-4002-95bb-e1393e75642e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:50.841077 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1bfc1-ad4f-4453-a6e2-c8d27fe22617 service nova] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:50.871630 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:50.931027 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:51.000336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1330660f-22c5-4002-95bb-e1393e75642e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:51.004051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1330660f-22c5-4002-95bb-e1393e75642e service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:51.024182 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 876/1746] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:51.103626 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5b6eb-6d74-492e-8dfa-b4241bc708b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:51.103626 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5b6eb-6d74-492e-8dfa-b4241bc708b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:51.103626 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5b6eb-6d74-492e-8dfa-b4241bc708b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:51.103941 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5b6eb-6d74-492e-8dfa-b4241bc708b8 admin admin] PATH_INFO: `/v3/users/9e307a7fb4394ff3812336fb8cc0e4ab` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:51.109353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a5b6eb-6d74-492e-8dfa-b4241bc708b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e307a7fb4394ff3812336fb8cc0e4ab)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:51.115616 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a5b6eb-6d74-492e-8dfa-b4241bc708b8 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:51.259042 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-f3a5b6eb-6d74-492e-8dfa-b4241bc708b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e307a7fb4394ff3812336fb8cc0e4ab'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:51.277310 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-f3a5b6eb-6d74-492e-8dfa-b4241bc708b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e307a7fb4394ff3812336fb8cc0e4ab'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:51.278512 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-f3a5b6eb-6d74-492e-8dfa-b4241bc708b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e307a7fb4394ff3812336fb8cc0e4ab'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:51.289909 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-f3a5b6eb-6d74-492e-8dfa-b4241bc708b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e307a7fb4394ff3812336fb8cc0e4ab'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:51.306449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-f3a5b6eb-6d74-492e-8dfa-b4241bc708b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e307a7fb4394ff3812336fb8cc0e4ab'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:51.336583 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 871/1747] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:45:50 2022] DELETE /identity/v3/users/9e307a7fb4394ff3812336fb8cc0e4ab => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:51.342483 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74ee44-2105-4405-a7be-2ca178410aa5 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:51.343007 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74ee44-2105-4405-a7be-2ca178410aa5 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:51.343347 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74ee44-2105-4405-a7be-2ca178410aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:51.390835 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-9a74ee44-2105-4405-a7be-2ca178410aa5 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:51.392521 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-9a74ee44-2105-4405-a7be-2ca178410aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:51.540911 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 877/1748] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:45:51.594068 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5b6eb-6d74-492e-8dfa-b4241bc708b8 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:51.598380 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-f3a5b6eb-6d74-492e-8dfa-b4241bc708b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:51.780701 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5b6eb-6d74-492e-8dfa-b4241bc708b8 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:51.805554 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a482ab-c14c-4b2b-86ac-8c3f347cec30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:51.806622 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a482ab-c14c-4b2b-86ac-8c3f347cec30 service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:51.810424 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a482ab-c14c-4b2b-86ac-8c3f347cec30 service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:51.810882 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a482ab-c14c-4b2b-86ac-8c3f347cec30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:51.856771 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a482ab-c14c-4b2b-86ac-8c3f347cec30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:51.858359 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a482ab-c14c-4b2b-86ac-8c3f347cec30 service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:51.889470 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 872/1749] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:53.392431 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74ee44-2105-4405-a7be-2ca178410aa5 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:53.393428 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-9a74ee44-2105-4405-a7be-2ca178410aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:53.411662 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74ee44-2105-4405-a7be-2ca178410aa5 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:53.433527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a91bec-5edb-483a-adf4-d0212f8becfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:53.434694 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a91bec-5edb-483a-adf4-d0212f8becfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:53.435052 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a91bec-5edb-483a-adf4-d0212f8becfa admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:53.435435 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a91bec-5edb-483a-adf4-d0212f8becfa admin admin] PATH_INFO: `/v3/projects/ecb3548713e744c69a4729856f837677` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:53.438013 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a91bec-5edb-483a-adf4-d0212f8becfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecb3548713e744c69a4729856f837677)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:53.457248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a91bec-5edb-483a-adf4-d0212f8becfa admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:53.580897 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b3a91bec-5edb-483a-adf4-d0212f8becfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecb3548713e744c69a4729856f837677', 'request_id': 'req-b3a91bec-5edb-483a-adf4-d0212f8becfa'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:53.597274 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b3a91bec-5edb-483a-adf4-d0212f8becfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecb3548713e744c69a4729856f837677', 'request_id': 'req-b3a91bec-5edb-483a-adf4-d0212f8becfa'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:53.614260 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b3a91bec-5edb-483a-adf4-d0212f8becfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecb3548713e744c69a4729856f837677', 'request_id': 'req-b3a91bec-5edb-483a-adf4-d0212f8becfa'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:53.616347 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a482ab-c14c-4b2b-86ac-8c3f347cec30 service neutron] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:53.619184 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-98a482ab-c14c-4b2b-86ac-8c3f347cec30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:53.636735 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b3a91bec-5edb-483a-adf4-d0212f8becfa admin admin] The token cache is being invalidate because project ecb3548713e744c69a4729856f837677 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:45:53.636735 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b3a91bec-5edb-483a-adf4-d0212f8becfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:53.637368 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b3a91bec-5edb-483a-adf4-d0212f8becfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:53.640705 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 878/1750] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:45:53 2022] DELETE /identity/v3/projects/ecb3548713e744c69a4729856f837677 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:53.643117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a482ab-c14c-4b2b-86ac-8c3f347cec30 service neutron] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:53.646219 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346224da-d749-4b8a-9af4-f45b639c57c0 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:53.646614 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346224da-d749-4b8a-9af4-f45b639c57c0 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:53.647161 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346224da-d749-4b8a-9af4-f45b639c57c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:53.676582 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-346224da-d749-4b8a-9af4-f45b639c57c0 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:53.677980 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-346224da-d749-4b8a-9af4-f45b639c57c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:53.742213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:53.783121 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:53.797943 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 879/1751] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:53.817470 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69dc4a-5cc8-48eb-8877-21afa06a7334 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:53.819095 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69dc4a-5cc8-48eb-8877-21afa06a7334 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:53.819387 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69dc4a-5cc8-48eb-8877-21afa06a7334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:53.839234 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d64d8f-ba90-4ef8-8b1c-0df747794c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:53.840226 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d64d8f-ba90-4ef8-8b1c-0df747794c01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:53.840725 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d64d8f-ba90-4ef8-8b1c-0df747794c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:53.842450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d64d8f-ba90-4ef8-8b1c-0df747794c01 admin admin] PATH_INFO: `/v3/projects/fc11b5c5b8d54b478df89e08aba0ad86` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:53.845080 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d64d8f-ba90-4ef8-8b1c-0df747794c01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc11b5c5b8d54b478df89e08aba0ad86)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:53.848630 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d64d8f-ba90-4ef8-8b1c-0df747794c01 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:53.875481 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-0c69dc4a-5cc8-48eb-8877-21afa06a7334 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:53.877898 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-0c69dc4a-5cc8-48eb-8877-21afa06a7334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:53.894773 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 880/1752] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:53.895866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69dc4a-5cc8-48eb-8877-21afa06a7334 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:53.919936 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69dc4a-5cc8-48eb-8877-21afa06a7334 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:53.931882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b796c6-a72a-4334-9dfa-7358a3ef2081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:53.933782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b796c6-a72a-4334-9dfa-7358a3ef2081 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:53.934463 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b796c6-a72a-4334-9dfa-7358a3ef2081 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:53.934756 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b796c6-a72a-4334-9dfa-7358a3ef2081 admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:53.935506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b796c6-a72a-4334-9dfa-7358a3ef2081 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:45:53.935823 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b796c6-a72a-4334-9dfa-7358a3ef2081 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:53.941421 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b796c6-a72a-4334-9dfa-7358a3ef2081 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:53.957519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-00d64d8f-ba90-4ef8-8b1c-0df747794c01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc11b5c5b8d54b478df89e08aba0ad86', 'request_id': 'req-00d64d8f-ba90-4ef8-8b1c-0df747794c01'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:53.980700 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 881/1753] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:45:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:45:53.986789 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-00d64d8f-ba90-4ef8-8b1c-0df747794c01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc11b5c5b8d54b478df89e08aba0ad86', 'request_id': 'req-00d64d8f-ba90-4ef8-8b1c-0df747794c01'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:53.988310 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b796c6-a72a-4334-9dfa-7358a3ef2081 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:54.004821 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-00d64d8f-ba90-4ef8-8b1c-0df747794c01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc11b5c5b8d54b478df89e08aba0ad86', 'request_id': 'req-00d64d8f-ba90-4ef8-8b1c-0df747794c01'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:54.012447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b796c6-a72a-4334-9dfa-7358a3ef2081 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:54.045693 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-00d64d8f-ba90-4ef8-8b1c-0df747794c01 admin admin] The token cache is being invalidate because project fc11b5c5b8d54b478df89e08aba0ad86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:45:54.046653 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-00d64d8f-ba90-4ef8-8b1c-0df747794c01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:54.047587 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-00d64d8f-ba90-4ef8-8b1c-0df747794c01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:54.054491 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 873/1754] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:45:53 2022] DELETE /identity/v3/projects/fc11b5c5b8d54b478df89e08aba0ad86 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:45:54.056823 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d64d8f-ba90-4ef8-8b1c-0df747794c01 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:54.059193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-00d64d8f-ba90-4ef8-8b1c-0df747794c01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:54.216564 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d64d8f-ba90-4ef8-8b1c-0df747794c01 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:54.219487 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46384d8e-6662-4be6-8883-4bb5d1de8caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:54.219768 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46384d8e-6662-4be6-8883-4bb5d1de8caf admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:54.222042 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46384d8e-6662-4be6-8883-4bb5d1de8caf admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:54.222965 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46384d8e-6662-4be6-8883-4bb5d1de8caf admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:54.223700 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46384d8e-6662-4be6-8883-4bb5d1de8caf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:54.226829 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46384d8e-6662-4be6-8883-4bb5d1de8caf admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:54.231533 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd25dc0a-4cb2-4801-aee4-871eada29171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:54.232600 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25dc0a-4cb2-4801-aee4-871eada29171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:54.232893 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25dc0a-4cb2-4801-aee4-871eada29171 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:54.233196 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25dc0a-4cb2-4801-aee4-871eada29171 admin admin] PATH_INFO: `/v3/users/eb88a84552d14bfca6627a38d7c2d385` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:54.235386 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd25dc0a-4cb2-4801-aee4-871eada29171 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb88a84552d14bfca6627a38d7c2d385)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:54.237437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd25dc0a-4cb2-4801-aee4-871eada29171 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:54.333645 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 882/1755] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:45:53 2022] POST /identity/v3/projects => generated 383 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:54.334622 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46384d8e-6662-4be6-8883-4bb5d1de8caf admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:54.348500 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-46384d8e-6662-4be6-8883-4bb5d1de8caf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:54.407311 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-cd25dc0a-4cb2-4801-aee4-871eada29171 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb88a84552d14bfca6627a38d7c2d385'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:54.434619 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-cd25dc0a-4cb2-4801-aee4-871eada29171 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb88a84552d14bfca6627a38d7c2d385'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:54.435840 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-cd25dc0a-4cb2-4801-aee4-871eada29171 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb88a84552d14bfca6627a38d7c2d385'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:54.447963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-cd25dc0a-4cb2-4801-aee4-871eada29171 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb88a84552d14bfca6627a38d7c2d385'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:54.455791 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:54.484009 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-cd25dc0a-4cb2-4801-aee4-871eada29171 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb88a84552d14bfca6627a38d7c2d385'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:54.499729 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 874/1756] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:45:54 2022] DELETE /identity/v3/users/eb88a84552d14bfca6627a38d7c2d385 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:54.548467 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46384d8e-6662-4be6-8883-4bb5d1de8caf admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:54.624717 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5592e-727e-489d-89ec-ea40652f693d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:54.625441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5592e-727e-489d-89ec-ea40652f693d admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:54.625868 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5592e-727e-489d-89ec-ea40652f693d admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:54.626147 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5592e-727e-489d-89ec-ea40652f693d admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:54.626879 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f5592e-727e-489d-89ec-ea40652f693d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:54.628778 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f5592e-727e-489d-89ec-ea40652f693d admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:54.646443 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:54.653117 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 883/1757] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:45:54 2022] POST /identity/v3/users => generated 292 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:54.657871 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd25dc0a-4cb2-4801-aee4-871eada29171 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:54.659730 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-cd25dc0a-4cb2-4801-aee4-871eada29171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:54.720633 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4bc1ad-0783-4450-b6d1-c0377ef9c691 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:54.721105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4bc1ad-0783-4450-b6d1-c0377ef9c691 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:54.721200 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4bc1ad-0783-4450-b6d1-c0377ef9c691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:54.728728 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd25dc0a-4cb2-4801-aee4-871eada29171 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:54.743245 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be4399-3ec6-46e3-a99b-526bc6695d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:54.744128 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be4399-3ec6-46e3-a99b-526bc6695d85 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:54.744521 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be4399-3ec6-46e3-a99b-526bc6695d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:54.744911 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be4399-3ec6-46e3-a99b-526bc6695d85 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:54.745826 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1be4399-3ec6-46e3-a99b-526bc6695d85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:54.747913 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1be4399-3ec6-46e3-a99b-526bc6695d85 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:54.766582 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-4c4bc1ad-0783-4450-b6d1-c0377ef9c691 None None] MFA Rules not processed for user `2d4da8cc84fe4716a316183a5a88096b`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:54.767807 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-4c4bc1ad-0783-4450-b6d1-c0377ef9c691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:54.777282 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 875/1758] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:54.779742 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be4399-3ec6-46e3-a99b-526bc6695d85 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:54.800896 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be4399-3ec6-46e3-a99b-526bc6695d85 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:54.808199 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8f768-8130-4d22-b833-547d5136311d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:54.808974 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8f768-8130-4d22-b833-547d5136311d admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:54.809296 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8f768-8130-4d22-b833-547d5136311d admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:54.809625 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8f768-8130-4d22-b833-547d5136311d admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:54.810410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8f768-8130-4d22-b833-547d5136311d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:54.812218 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8f768-8130-4d22-b833-547d5136311d admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:54.827752 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 876/1759] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:54.828669 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8f768-8130-4d22-b833-547d5136311d admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:54.839747 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8f768-8130-4d22-b833-547d5136311d admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:54.845035 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a5bf6b-d5ad-43c0-9a7f-768305bff8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:54.846210 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5bf6b-d5ad-43c0-9a7f-768305bff8c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:54.846498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5bf6b-d5ad-43c0-9a7f-768305bff8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:54.847266 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5bf6b-d5ad-43c0-9a7f-768305bff8c7 admin admin] PATH_INFO: `/v3/projects/f280861f92d24260800f66464e691f57/users/6dc6496b22704b2991737c5e648c1538/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:54.865202 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 884/1760] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:54 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:45:54.866160 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4bc1ad-0783-4450-b6d1-c0377ef9c691 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:54.869534 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-4c4bc1ad-0783-4450-b6d1-c0377ef9c691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:54.874554 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a5bf6b-d5ad-43c0-9a7f-768305bff8c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f280861f92d24260800f66464e691f57, user_id=6dc6496b22704b2991737c5e648c1538, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:54.878812 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a5bf6b-d5ad-43c0-9a7f-768305bff8c7 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:54.903349 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4bc1ad-0783-4450-b6d1-c0377ef9c691 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:54.913352 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a3856-638a-4a3d-b39a-be58458137d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:54.914089 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a3856-638a-4a3d-b39a-be58458137d4 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:54.914497 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a3856-638a-4a3d-b39a-be58458137d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:54.914785 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a3856-638a-4a3d-b39a-be58458137d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:54.915539 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a3856-638a-4a3d-b39a-be58458137d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:54.918169 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a3856-638a-4a3d-b39a-be58458137d4 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:54.921160 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 877/1761] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:45:54 2022] PUT /identity/v3/projects/f280861f92d24260800f66464e691f57/users/6dc6496b22704b2991737c5e648c1538/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:54.935581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96da8e-2580-4bea-b46d-6afac3be7a23 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:54.936416 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96da8e-2580-4bea-b46d-6afac3be7a23 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:54.936906 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96da8e-2580-4bea-b46d-6afac3be7a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:54.969539 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 885/1762] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:45:54 2022] POST /identity/v3/projects => generated 405 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:54.974080 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a3856-638a-4a3d-b39a-be58458137d4 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:54.987180 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-5f96da8e-2580-4bea-b46d-6afac3be7a23 None None] MFA Rules not processed for user `6dc6496b22704b2991737c5e648c1538`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:54.989639 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-5f96da8e-2580-4bea-b46d-6afac3be7a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:55.047311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a3856-638a-4a3d-b39a-be58458137d4 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:55.077209 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b84096-c6eb-4b79-8090-91e651b5f46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:55.078502 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b84096-c6eb-4b79-8090-91e651b5f46c admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:55.078798 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b84096-c6eb-4b79-8090-91e651b5f46c admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:55.079137 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b84096-c6eb-4b79-8090-91e651b5f46c admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:55.079856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b84096-c6eb-4b79-8090-91e651b5f46c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:55.081844 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b84096-c6eb-4b79-8090-91e651b5f46c admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:55.083002 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 878/1763] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:55.087652 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f96da8e-2580-4bea-b46d-6afac3be7a23 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:55.105037 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f96da8e-2580-4bea-b46d-6afac3be7a23 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:55.115328 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 886/1764] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:45:54 2022] POST /identity/v3/users => generated 309 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:55.120880 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b84096-c6eb-4b79-8090-91e651b5f46c admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:55.126428 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51683c4c-2681-4a27-babc-cbe303600988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:55.127323 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51683c4c-2681-4a27-babc-cbe303600988 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:55.127642 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51683c4c-2681-4a27-babc-cbe303600988 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:55.128044 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51683c4c-2681-4a27-babc-cbe303600988 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:55.128835 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51683c4c-2681-4a27-babc-cbe303600988 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:55.131140 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51683c4c-2681-4a27-babc-cbe303600988 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:55.134949 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b84096-c6eb-4b79-8090-91e651b5f46c admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:55.165649 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e429987-fa56-45b8-b248-594e2de030d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:55.166550 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e429987-fa56-45b8-b248-594e2de030d3 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:55.167003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e429987-fa56-45b8-b248-594e2de030d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:55.167287 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e429987-fa56-45b8-b248-594e2de030d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:55.168030 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e429987-fa56-45b8-b248-594e2de030d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:55.169703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e429987-fa56-45b8-b248-594e2de030d3 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:55.177205 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 879/1765] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:45:55 2022] POST /identity/v3/projects => generated 383 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:45:55.177205 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51683c4c-2681-4a27-babc-cbe303600988 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:55.179499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-51683c4c-2681-4a27-babc-cbe303600988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:55.185710 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 887/1766] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:55.186454 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e429987-fa56-45b8-b248-594e2de030d3 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:55.197137 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e429987-fa56-45b8-b248-594e2de030d3 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:55.248964 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:55.266467 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d95c1-b4bc-4af1-a933-a8b8b7e134d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:55.267145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d95c1-b4bc-4af1-a933-a8b8b7e134d6 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:55.267431 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d95c1-b4bc-4af1-a933-a8b8b7e134d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:55.267707 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d95c1-b4bc-4af1-a933-a8b8b7e134d6 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:55.268403 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2d95c1-b4bc-4af1-a933-a8b8b7e134d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:55.270420 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2d95c1-b4bc-4af1-a933-a8b8b7e134d6 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:55.293752 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51683c4c-2681-4a27-babc-cbe303600988 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:55.300817 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 888/1767] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:45:55 2022] POST /identity/v3/users => generated 298 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:55.301938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d95c1-b4bc-4af1-a933-a8b8b7e134d6 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:55.308888 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94ae5e-5842-447b-baa8-7933f67fda30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:55.309842 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94ae5e-5842-447b-baa8-7933f67fda30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:55.310129 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94ae5e-5842-447b-baa8-7933f67fda30 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:55.310408 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94ae5e-5842-447b-baa8-7933f67fda30 admin admin] PATH_INFO: `/v3/projects/2a56989b6c864610bceb0192c52d8cef` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:55.311622 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c94ae5e-5842-447b-baa8-7933f67fda30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a56989b6c864610bceb0192c52d8cef)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:55.313599 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c94ae5e-5842-447b-baa8-7933f67fda30 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:55.314094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d95c1-b4bc-4af1-a933-a8b8b7e134d6 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:55.317417 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:55.319442 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc9c4d-cc3c-4d12-85ab-8999b924b69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:55.320883 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc9c4d-cc3c-4d12-85ab-8999b924b69c admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:55.321380 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc9c4d-cc3c-4d12-85ab-8999b924b69c admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:55.321663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc9c4d-cc3c-4d12-85ab-8999b924b69c admin admin] PATH_INFO: `/v3/projects/9ff9e1b18a2747dbaf70558c24126667/users/ce5b1632fdb44e9dad331b1ed19d9b06/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:55.339794 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fc9c4d-cc3c-4d12-85ab-8999b924b69c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ff9e1b18a2747dbaf70558c24126667, user_id=ce5b1632fdb44e9dad331b1ed19d9b06, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:55.344225 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fc9c4d-cc3c-4d12-85ab-8999b924b69c admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:55.377042 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 889/1768] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:45:55 2022] PUT /identity/v3/projects/9ff9e1b18a2747dbaf70558c24126667/users/ce5b1632fdb44e9dad331b1ed19d9b06/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:55.377042 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc9c4d-cc3c-4d12-85ab-8999b924b69c admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:55.377615 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-1c94ae5e-5842-447b-baa8-7933f67fda30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a56989b6c864610bceb0192c52d8cef', 'request_id': 'req-1c94ae5e-5842-447b-baa8-7933f67fda30'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:55.379569 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-1c94ae5e-5842-447b-baa8-7933f67fda30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a56989b6c864610bceb0192c52d8cef', 'request_id': 'req-1c94ae5e-5842-447b-baa8-7933f67fda30'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:55.390318 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-1c94ae5e-5842-447b-baa8-7933f67fda30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a56989b6c864610bceb0192c52d8cef', 'request_id': 'req-1c94ae5e-5842-447b-baa8-7933f67fda30'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:55.404235 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-1c94ae5e-5842-447b-baa8-7933f67fda30 admin admin] The token cache is being invalidate because project 2a56989b6c864610bceb0192c52d8cef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:45:55.404631 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-1c94ae5e-5842-447b-baa8-7933f67fda30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:55.405422 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-1c94ae5e-5842-447b-baa8-7933f67fda30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:55.410259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 880/1769] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:45:55 2022] DELETE /identity/v3/projects/2a56989b6c864610bceb0192c52d8cef => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:55.410259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94ae5e-5842-447b-baa8-7933f67fda30 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:55.410259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-1c94ae5e-5842-447b-baa8-7933f67fda30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:55.443124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc9c4d-cc3c-4d12-85ab-8999b924b69c admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:55.448972 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dedee7-a356-468e-9bc6-f6b6c9d0f7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:55.449672 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dedee7-a356-468e-9bc6-f6b6c9d0f7c8 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:55.449964 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dedee7-a356-468e-9bc6-f6b6c9d0f7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:55.450243 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dedee7-a356-468e-9bc6-f6b6c9d0f7c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:55.450976 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dedee7-a356-468e-9bc6-f6b6c9d0f7c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:55.452585 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dedee7-a356-468e-9bc6-f6b6c9d0f7c8 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:55.466733 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 890/1770] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:55.466733 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd9f76-1bcb-4b22-8eed-a187cd7551a1 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:55.466733 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd9f76-1bcb-4b22-8eed-a187cd7551a1 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:55.466733 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd9f76-1bcb-4b22-8eed-a187cd7551a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:55.485352 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-02cd9f76-1bcb-4b22-8eed-a187cd7551a1 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:55.486614 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-02cd9f76-1bcb-4b22-8eed-a187cd7551a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:55.487469 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94ae5e-5842-447b-baa8-7933f67fda30 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:55.493077 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c2e4c-d953-4dad-ac3e-a0305cc9cf80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:55.493760 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c2e4c-d953-4dad-ac3e-a0305cc9cf80 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:55.494202 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c2e4c-d953-4dad-ac3e-a0305cc9cf80 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:55.494485 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c2e4c-d953-4dad-ac3e-a0305cc9cf80 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:55.495341 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48c2e4c-d953-4dad-ac3e-a0305cc9cf80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:55.496518 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 891/1771] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:55.497210 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48c2e4c-d953-4dad-ac3e-a0305cc9cf80 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:55.497344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd9f76-1bcb-4b22-8eed-a187cd7551a1 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:55.498896 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-02cd9f76-1bcb-4b22-8eed-a187cd7551a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:55.508477 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 881/1772] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:55.509797 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2dab8-7852-452d-a51a-3cea46500ca8 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:55.510118 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2dab8-7852-452d-a51a-3cea46500ca8 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:55.510400 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2dab8-7852-452d-a51a-3cea46500ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:55.516331 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd9f76-1bcb-4b22-8eed-a187cd7551a1 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:55.522034 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477601b-2d3e-4c17-925d-385a7e15239d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:55.523935 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477601b-2d3e-4c17-925d-385a7e15239d admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:55.524496 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477601b-2d3e-4c17-925d-385a7e15239d admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:55.524800 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477601b-2d3e-4c17-925d-385a7e15239d admin admin] PATH_INFO: `/v3/projects/5154e8024ab74ee7bb9740ee6d5417d1/users/9293c5a077404693b276048986c7eaea/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:55.529496 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-fac2dab8-7852-452d-a51a-3cea46500ca8 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:55.530876 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-fac2dab8-7852-452d-a51a-3cea46500ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:55.543071 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 882/1773] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:55.543201 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9477601b-2d3e-4c17-925d-385a7e15239d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5154e8024ab74ee7bb9740ee6d5417d1, user_id=9293c5a077404693b276048986c7eaea, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:55.543283 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2dab8-7852-452d-a51a-3cea46500ca8 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:55.543353 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9477601b-2d3e-4c17-925d-385a7e15239d admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:55.553256 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2dab8-7852-452d-a51a-3cea46500ca8 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:55.557990 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 892/1774] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:45:55 2022] PUT /identity/v3/projects/5154e8024ab74ee7bb9740ee6d5417d1/users/9293c5a077404693b276048986c7eaea/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:55.559003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477601b-2d3e-4c17-925d-385a7e15239d admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:55.610146 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a427f30-141c-459c-bee4-b5c5f510d6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:55.611051 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a427f30-141c-459c-bee4-b5c5f510d6e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:55.611336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a427f30-141c-459c-bee4-b5c5f510d6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:55.611612 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a427f30-141c-459c-bee4-b5c5f510d6e6 admin admin] PATH_INFO: `/v3/projects/9ff9e1b18a2747dbaf70558c24126667/users/ce5b1632fdb44e9dad331b1ed19d9b06/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:55.613692 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a427f30-141c-459c-bee4-b5c5f510d6e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ff9e1b18a2747dbaf70558c24126667, user_id=ce5b1632fdb44e9dad331b1ed19d9b06, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:55.616379 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a427f30-141c-459c-bee4-b5c5f510d6e6 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:55.627343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 883/1775] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:45:55 2022] PUT /identity/v3/projects/9ff9e1b18a2747dbaf70558c24126667/users/ce5b1632fdb44e9dad331b1ed19d9b06/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:55.628140 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a427f30-141c-459c-bee4-b5c5f510d6e6 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:55.629683 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477601b-2d3e-4c17-925d-385a7e15239d admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:55.649290 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:55.653437 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:55.700898 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd4678-63a6-4422-ae8f-2a4abb4bdd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:55.701642 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd4678-63a6-4422-ae8f-2a4abb4bdd65 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:55.701878 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd4678-63a6-4422-ae8f-2a4abb4bdd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:55.702159 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd4678-63a6-4422-ae8f-2a4abb4bdd65 admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:55.702894 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fd4678-63a6-4422-ae8f-2a4abb4bdd65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:45:55.703893 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fd4678-63a6-4422-ae8f-2a4abb4bdd65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:55.704812 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fd4678-63a6-4422-ae8f-2a4abb4bdd65 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:55.707864 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a427f30-141c-459c-bee4-b5c5f510d6e6 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:55.713862 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d2bc36-7595-4678-93ab-7e4cc23b705d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:55.714593 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2bc36-7595-4678-93ab-7e4cc23b705d admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:55.714884 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2bc36-7595-4678-93ab-7e4cc23b705d admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:55.715234 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2bc36-7595-4678-93ab-7e4cc23b705d admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:55.716201 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d2bc36-7595-4678-93ab-7e4cc23b705d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:55.718075 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d2bc36-7595-4678-93ab-7e4cc23b705d admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:55.723242 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 893/1776] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:45:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:45:55.724415 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd4678-63a6-4422-ae8f-2a4abb4bdd65 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:55.735057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 884/1777] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:55.735673 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd4678-63a6-4422-ae8f-2a4abb4bdd65 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:55.739525 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d2bc36-7595-4678-93ab-7e4cc23b705d admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:55.740691 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9853bad5-51b3-45dc-88f5-2279627a0d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:55.741420 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853bad5-51b3-45dc-88f5-2279627a0d29 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:55.741686 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853bad5-51b3-45dc-88f5-2279627a0d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:55.742457 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853bad5-51b3-45dc-88f5-2279627a0d29 admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:55.743160 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9853bad5-51b3-45dc-88f5-2279627a0d29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:55.745080 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9853bad5-51b3-45dc-88f5-2279627a0d29 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:55.755615 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d2bc36-7595-4678-93ab-7e4cc23b705d admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:55.764807 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9423d2-cb23-4074-9683-206e814a89b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:55.766069 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9423d2-cb23-4074-9683-206e814a89b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:55.766415 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9423d2-cb23-4074-9683-206e814a89b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:55.766916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9423d2-cb23-4074-9683-206e814a89b7 admin admin] PATH_INFO: `/v3/projects/5154e8024ab74ee7bb9740ee6d5417d1/users/9293c5a077404693b276048986c7eaea/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:55.769360 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9423d2-cb23-4074-9683-206e814a89b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5154e8024ab74ee7bb9740ee6d5417d1, user_id=9293c5a077404693b276048986c7eaea, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:55.772030 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9423d2-cb23-4074-9683-206e814a89b7 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:55.782975 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 894/1778] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:45:55 2022] POST /identity/v3/projects => generated 381 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:55.783460 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 885/1779] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:45:55 2022] PUT /identity/v3/projects/5154e8024ab74ee7bb9740ee6d5417d1/users/9293c5a077404693b276048986c7eaea/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:45:55.790855 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9853bad5-51b3-45dc-88f5-2279627a0d29 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:55.794089 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59e271-9b90-4e24-a2e9-971c95565439 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:55.794436 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59e271-9b90-4e24-a2e9-971c95565439 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:55.794730 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59e271-9b90-4e24-a2e9-971c95565439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:55.840149 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-db59e271-9b90-4e24-a2e9-971c95565439 None None] MFA Rules not processed for user `9293c5a077404693b276048986c7eaea`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:55.841451 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-db59e271-9b90-4e24-a2e9-971c95565439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:55.872504 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9853bad5-51b3-45dc-88f5-2279627a0d29 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:55.878385 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1921fa83-2d6a-43e8-960f-e42ab43b4b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:55.879177 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921fa83-2d6a-43e8-960f-e42ab43b4b95 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:55.879440 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921fa83-2d6a-43e8-960f-e42ab43b4b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:55.879723 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921fa83-2d6a-43e8-960f-e42ab43b4b95 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:55.880428 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1921fa83-2d6a-43e8-960f-e42ab43b4b95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:55.883270 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1921fa83-2d6a-43e8-960f-e42ab43b4b95 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:55.910782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 895/1780] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:45:55 2022] POST /identity/v3/users => generated 291 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:55.915527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1921fa83-2d6a-43e8-960f-e42ab43b4b95 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:55.926861 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1921fa83-2d6a-43e8-960f-e42ab43b4b95 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:55.932651 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681f4b0-34d2-499c-9c37-3c6e51f391b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:55.933426 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681f4b0-34d2-499c-9c37-3c6e51f391b3 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:55.933779 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681f4b0-34d2-499c-9c37-3c6e51f391b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:55.934133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681f4b0-34d2-499c-9c37-3c6e51f391b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:55.934872 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1681f4b0-34d2-499c-9c37-3c6e51f391b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:55.939554 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 886/1781] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:55 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:45:55.939705 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1681f4b0-34d2-499c-9c37-3c6e51f391b3 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:55.949829 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db59e271-9b90-4e24-a2e9-971c95565439 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:55.951295 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-db59e271-9b90-4e24-a2e9-971c95565439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:55.957364 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 896/1782] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:45:55.959205 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681f4b0-34d2-499c-9c37-3c6e51f391b3 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:55.973420 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681f4b0-34d2-499c-9c37-3c6e51f391b3 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:55.980940 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b1e25-13ac-473e-af36-a25d40b80b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:55.981727 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b1e25-13ac-473e-af36-a25d40b80b16 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:55.982019 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b1e25-13ac-473e-af36-a25d40b80b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:55.982300 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b1e25-13ac-473e-af36-a25d40b80b16 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:55.983033 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580b1e25-13ac-473e-af36-a25d40b80b16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:55.984644 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580b1e25-13ac-473e-af36-a25d40b80b16 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:56.005181 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 897/1783] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:56.009448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b1e25-13ac-473e-af36-a25d40b80b16 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:56.025650 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db59e271-9b90-4e24-a2e9-971c95565439 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:56.027216 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b1e25-13ac-473e-af36-a25d40b80b16 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:56.029995 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905d442-bdc9-49fb-9b97-af28668dcaee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:56.031220 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905d442-bdc9-49fb-9b97-af28668dcaee service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:56.031577 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905d442-bdc9-49fb-9b97-af28668dcaee service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:56.031845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905d442-bdc9-49fb-9b97-af28668dcaee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:56.033119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94ce9cb-f984-4731-9d2b-ff6982f23fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:56.033963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-c905d442-bdc9-49fb-9b97-af28668dcaee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:56.034256 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ce9cb-f984-4731-9d2b-ff6982f23fb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:56.034776 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ce9cb-f984-4731-9d2b-ff6982f23fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:56.035029 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ce9cb-f984-4731-9d2b-ff6982f23fb4 admin admin] PATH_INFO: `/v3/projects/ba2955d90adb476ba558e647835eb50a/users/bb61482b630a41ddbec3857af513a52a/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:56.057456 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94ce9cb-f984-4731-9d2b-ff6982f23fb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba2955d90adb476ba558e647835eb50a, user_id=bb61482b630a41ddbec3857af513a52a, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:56.061710 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94ce9cb-f984-4731-9d2b-ff6982f23fb4 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:56.086861 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:56.101974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 898/1784] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:45:56 2022] PUT /identity/v3/projects/ba2955d90adb476ba558e647835eb50a/users/bb61482b630a41ddbec3857af513a52a/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:56.112712 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d2701-ccf1-451f-b3c2-d9a0e756bbce None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:56.113142 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d2701-ccf1-451f-b3c2-d9a0e756bbce None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:56.113464 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d2701-ccf1-451f-b3c2-d9a0e756bbce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:56.119349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905d442-bdc9-49fb-9b97-af28668dcaee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:56.121476 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905d442-bdc9-49fb-9b97-af28668dcaee service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:56.136368 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 887/1785] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:56.146874 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-8b9d2701-ccf1-451f-b3c2-d9a0e756bbce None None] MFA Rules not processed for user `bb61482b630a41ddbec3857af513a52a`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:56.149263 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-8b9d2701-ccf1-451f-b3c2-d9a0e756bbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:56.192407 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:56.243818 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 899/1786] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:56.244061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905d442-bdc9-49fb-9b97-af28668dcaee service cinder] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:56.258080 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905d442-bdc9-49fb-9b97-af28668dcaee service cinder] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:56.356190 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7692a-b1cd-469f-968c-8362fbb51a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:56.361277 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7692a-b1cd-469f-968c-8362fbb51a39 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:56.363174 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7692a-b1cd-469f-968c-8362fbb51a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:56.363633 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7692a-b1cd-469f-968c-8362fbb51a39 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:56.365451 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c7692a-b1cd-469f-968c-8362fbb51a39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:56.368227 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c7692a-b1cd-469f-968c-8362fbb51a39 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:56.416834 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 888/1787] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:45:56 2022] POST /identity/v3/projects => generated 387 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:45:56.421214 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d2701-ccf1-451f-b3c2-d9a0e756bbce None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:56.441136 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d2701-ccf1-451f-b3c2-d9a0e756bbce None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:56.525758 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2a377-e5bf-4b2d-9744-35a52bb8065f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:56.526454 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2a377-e5bf-4b2d-9744-35a52bb8065f admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:56.526740 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2a377-e5bf-4b2d-9744-35a52bb8065f admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:56.527097 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2a377-e5bf-4b2d-9744-35a52bb8065f admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:56.528067 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c2a377-e5bf-4b2d-9744-35a52bb8065f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:56.530166 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c2a377-e5bf-4b2d-9744-35a52bb8065f admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:56.566243 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 900/1788] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:45:56 2022] POST /identity/v3/users => generated 300 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:45:56.570611 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7692a-b1cd-469f-968c-8362fbb51a39 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:56.588842 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7692a-b1cd-469f-968c-8362fbb51a39 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:56.599768 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66089f26-128f-43fe-aa60-6471e9b1707e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:56.600462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66089f26-128f-43fe-aa60-6471e9b1707e admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:56.600745 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66089f26-128f-43fe-aa60-6471e9b1707e admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:56.601055 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66089f26-128f-43fe-aa60-6471e9b1707e admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:56.602399 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66089f26-128f-43fe-aa60-6471e9b1707e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:56.606957 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66089f26-128f-43fe-aa60-6471e9b1707e admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:56.663320 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2a377-e5bf-4b2d-9744-35a52bb8065f admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:56.664246 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 889/1789] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:45:56.682743 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2a377-e5bf-4b2d-9744-35a52bb8065f admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:56.697092 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe90aa-f438-40ac-861d-681265708287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:56.698173 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe90aa-f438-40ac-861d-681265708287 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:56.699878 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe90aa-f438-40ac-861d-681265708287 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:56.700278 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe90aa-f438-40ac-861d-681265708287 admin admin] PATH_INFO: `/v3/projects/17f971604c174b028821c9382de44729/users/fc5cfe760daf4ea890b478f3ed031d0a/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:56.749734 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afe90aa-f438-40ac-861d-681265708287 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17f971604c174b028821c9382de44729, user_id=fc5cfe760daf4ea890b478f3ed031d0a, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:56.753968 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afe90aa-f438-40ac-861d-681265708287 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:56.788004 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 901/1790] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:45:56 2022] PUT /identity/v3/projects/17f971604c174b028821c9382de44729/users/fc5cfe760daf4ea890b478f3ed031d0a/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:45:56.791737 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66089f26-128f-43fe-aa60-6471e9b1707e admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:56.807636 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337eb254-ccb9-48df-9841-531ea001b860 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:56.807933 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337eb254-ccb9-48df-9841-531ea001b860 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:56.808214 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337eb254-ccb9-48df-9841-531ea001b860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:56.817159 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66089f26-128f-43fe-aa60-6471e9b1707e admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:56.855683 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-337eb254-ccb9-48df-9841-531ea001b860 None None] MFA Rules not processed for user `ce5b1632fdb44e9dad331b1ed19d9b06`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:56.856915 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-337eb254-ccb9-48df-9841-531ea001b860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:56.897644 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e8d44-98c3-4c78-8a3e-75f14814d028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:56.898919 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e8d44-98c3-4c78-8a3e-75f14814d028 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:56.899270 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e8d44-98c3-4c78-8a3e-75f14814d028 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:56.899615 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e8d44-98c3-4c78-8a3e-75f14814d028 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:56.900482 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213e8d44-98c3-4c78-8a3e-75f14814d028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:56.902196 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213e8d44-98c3-4c78-8a3e-75f14814d028 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:56.916833 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 890/1791] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:45:56.918302 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e8d44-98c3-4c78-8a3e-75f14814d028 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:56.931194 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e8d44-98c3-4c78-8a3e-75f14814d028 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:56.936979 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bcc045-128c-4ac3-9dcb-3a18280ec89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:56.937793 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bcc045-128c-4ac3-9dcb-3a18280ec89e admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:56.938136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bcc045-128c-4ac3-9dcb-3a18280ec89e admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:56.938476 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bcc045-128c-4ac3-9dcb-3a18280ec89e admin admin] PATH_INFO: `/v3/projects/17f971604c174b028821c9382de44729/users/fc5cfe760daf4ea890b478f3ed031d0a/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:56.940653 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bcc045-128c-4ac3-9dcb-3a18280ec89e admin admin] RBAC: Authorizing `identity:create_grant(project_id=17f971604c174b028821c9382de44729, user_id=fc5cfe760daf4ea890b478f3ed031d0a, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:56.941592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 902/1792] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:56 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:45:56.943587 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bcc045-128c-4ac3-9dcb-3a18280ec89e admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:56.954962 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 891/1793] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:45:56 2022] PUT /identity/v3/projects/17f971604c174b028821c9382de44729/users/fc5cfe760daf4ea890b478f3ed031d0a/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:45:56.966285 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb399a-76fb-4f44-ab1d-6bd0bb0d047c None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:56.966573 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb399a-76fb-4f44-ab1d-6bd0bb0d047c None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:56.966849 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb399a-76fb-4f44-ab1d-6bd0bb0d047c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:56.994475 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-83fb399a-76fb-4f44-ab1d-6bd0bb0d047c None None] MFA Rules not processed for user `fc5cfe760daf4ea890b478f3ed031d0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:56.995794 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-83fb399a-76fb-4f44-ab1d-6bd0bb0d047c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:57.004544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:57.068091 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 903/1794] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:56 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:57.079175 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bcc045-128c-4ac3-9dcb-3a18280ec89e admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:57.080838 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-b8bcc045-128c-4ac3-9dcb-3a18280ec89e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:57.124538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:57.147547 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bcc045-128c-4ac3-9dcb-3a18280ec89e admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:57.151893 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e20f2e2-458b-4512-8a72-aae503e42ad9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:57.152670 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20f2e2-458b-4512-8a72-aae503e42ad9 service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:57.152957 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20f2e2-458b-4512-8a72-aae503e42ad9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:57.153268 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20f2e2-458b-4512-8a72-aae503e42ad9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:57.163493 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e20f2e2-458b-4512-8a72-aae503e42ad9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:57.165503 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e20f2e2-458b-4512-8a72-aae503e42ad9 service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:57.180522 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 892/1795] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:57 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:57.311500 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb399a-76fb-4f44-ab1d-6bd0bb0d047c None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:57.392946 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb399a-76fb-4f44-ab1d-6bd0bb0d047c None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:57.397396 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b3bf74-9fac-49cf-8ffd-d65e6788b2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:57.398956 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3bf74-9fac-49cf-8ffd-d65e6788b2ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:57.399351 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3bf74-9fac-49cf-8ffd-d65e6788b2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:57.399734 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3bf74-9fac-49cf-8ffd-d65e6788b2ba admin admin] PATH_INFO: `/v3/users/bb61482b630a41ddbec3857af513a52a` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:57.401111 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b3bf74-9fac-49cf-8ffd-d65e6788b2ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb61482b630a41ddbec3857af513a52a)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:57.403247 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b3bf74-9fac-49cf-8ffd-d65e6788b2ba admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:57.554326 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c6b3bf74-9fac-49cf-8ffd-d65e6788b2ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb61482b630a41ddbec3857af513a52a'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:57.609377 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c6b3bf74-9fac-49cf-8ffd-d65e6788b2ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb61482b630a41ddbec3857af513a52a'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:57.618408 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c6b3bf74-9fac-49cf-8ffd-d65e6788b2ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb61482b630a41ddbec3857af513a52a'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:57.619041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c6b3bf74-9fac-49cf-8ffd-d65e6788b2ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb61482b630a41ddbec3857af513a52a'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:57.630645 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c6b3bf74-9fac-49cf-8ffd-d65e6788b2ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb61482b630a41ddbec3857af513a52a'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:57.637172 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 904/1796] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:45:57 2022] DELETE /identity/v3/users/bb61482b630a41ddbec3857af513a52a => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:57.641647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0878c-bd75-4f43-a3e2-b1ac94e0c76a None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:57.641936 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0878c-bd75-4f43-a3e2-b1ac94e0c76a None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:57.642217 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0878c-bd75-4f43-a3e2-b1ac94e0c76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:57.661066 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-30f0878c-bd75-4f43-a3e2-b1ac94e0c76a None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:57.662261 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-30f0878c-bd75-4f43-a3e2-b1ac94e0c76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:57.721978 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 893/1797] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:57.732664 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b3bf74-9fac-49cf-8ffd-d65e6788b2ba admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:57.734153 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-c6b3bf74-9fac-49cf-8ffd-d65e6788b2ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:57.763720 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:57.799510 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b3bf74-9fac-49cf-8ffd-d65e6788b2ba admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:57.800052 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7491a3e-1123-4f13-aa18-216c741ceeb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:57.800699 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7491a3e-1123-4f13-aa18-216c741ceeb4 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:57.800980 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7491a3e-1123-4f13-aa18-216c741ceeb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:57.801540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7491a3e-1123-4f13-aa18-216c741ceeb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:57.810353 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7491a3e-1123-4f13-aa18-216c741ceeb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:57.811755 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7491a3e-1123-4f13-aa18-216c741ceeb4 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:57.821434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 905/1798] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:58.134700 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2635c-e300-4c5b-9d24-6051698372b7 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:58.135263 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2635c-e300-4c5b-9d24-6051698372b7 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:58.135263 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2635c-e300-4c5b-9d24-6051698372b7 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:58.137184 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 894/1799] 10.222.0.20 () {54 vars in 921 bytes} [Wed Jul 6 13:45:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:45:58.146677 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890efcf-fff8-4c81-8b44-5a1a5ea13a2d None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:58.146980 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890efcf-fff8-4c81-8b44-5a1a5ea13a2d None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:58.147266 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890efcf-fff8-4c81-8b44-5a1a5ea13a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:58.166712 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-f890efcf-fff8-4c81-8b44-5a1a5ea13a2d None None] MFA Rules not processed for user `890d2ae150d647a18dae25159cf97484`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:58.167939 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-f890efcf-fff8-4c81-8b44-5a1a5ea13a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:58.259940 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 906/1800] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Jul 6 13:45:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:58.278176 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2635c-e300-4c5b-9d24-6051698372b7 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:58.279951 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-13a2635c-e300-4c5b-9d24-6051698372b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:58.322111 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2635c-e300-4c5b-9d24-6051698372b7 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:58.323850 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890efcf-fff8-4c81-8b44-5a1a5ea13a2d None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:58.325729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-f890efcf-fff8-4c81-8b44-5a1a5ea13a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:58.327078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519e35d-3b92-4abe-bd8d-f359e23cc95c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '890d2ae150d647a18dae25159cf97484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:58.327747 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519e35d-3b92-4abe-bd8d-f359e23cc95c service placement] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:58.328035 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519e35d-3b92-4abe-bd8d-f359e23cc95c service placement] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:58.328315 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519e35d-3b92-4abe-bd8d-f359e23cc95c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:58.343426 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890efcf-fff8-4c81-8b44-5a1a5ea13a2d None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:58.346619 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519e35d-3b92-4abe-bd8d-f359e23cc95c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:58.348745 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519e35d-3b92-4abe-bd8d-f359e23cc95c service placement] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:58.349234 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21edc256-4ba6-462b-a766-d25a73a04754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:58.350194 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21edc256-4ba6-462b-a766-d25a73a04754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:58.351864 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21edc256-4ba6-462b-a766-d25a73a04754 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:58.352089 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21edc256-4ba6-462b-a766-d25a73a04754 admin admin] PATH_INFO: `/v3/projects/ba2955d90adb476ba558e647835eb50a` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:58.354008 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21edc256-4ba6-462b-a766-d25a73a04754 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba2955d90adb476ba558e647835eb50a)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:58.356318 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21edc256-4ba6-462b-a766-d25a73a04754 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:58.381857 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 895/1801] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:58.424581 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-21edc256-4ba6-462b-a766-d25a73a04754 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba2955d90adb476ba558e647835eb50a', 'request_id': 'req-21edc256-4ba6-462b-a766-d25a73a04754'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:58.431518 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-21edc256-4ba6-462b-a766-d25a73a04754 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba2955d90adb476ba558e647835eb50a', 'request_id': 'req-21edc256-4ba6-462b-a766-d25a73a04754'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:58.439676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-21edc256-4ba6-462b-a766-d25a73a04754 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba2955d90adb476ba558e647835eb50a', 'request_id': 'req-21edc256-4ba6-462b-a766-d25a73a04754'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:58.451949 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-21edc256-4ba6-462b-a766-d25a73a04754 admin admin] The token cache is being invalidate because project ba2955d90adb476ba558e647835eb50a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:45:58.452242 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-21edc256-4ba6-462b-a766-d25a73a04754 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:58.452890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-21edc256-4ba6-462b-a766-d25a73a04754 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:58.454905 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 907/1802] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:45:58 2022] DELETE /identity/v3/projects/ba2955d90adb476ba558e647835eb50a => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:58.463623 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519e35d-3b92-4abe-bd8d-f359e23cc95c service placement] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:58.476433 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519e35d-3b92-4abe-bd8d-f359e23cc95c service placement] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:58.553060 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e558524-2800-4612-93e3-dc799cb2e13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:58.553060 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e558524-2800-4612-93e3-dc799cb2e13d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:58.553378 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e558524-2800-4612-93e3-dc799cb2e13d admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:58.553561 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e558524-2800-4612-93e3-dc799cb2e13d admin admin] PATH_INFO: `/v3/users/fc5cfe760daf4ea890b478f3ed031d0a` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:58.555770 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e558524-2800-4612-93e3-dc799cb2e13d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc5cfe760daf4ea890b478f3ed031d0a)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:58.557918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e558524-2800-4612-93e3-dc799cb2e13d admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:58.638694 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-3e558524-2800-4612-93e3-dc799cb2e13d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc5cfe760daf4ea890b478f3ed031d0a'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:58.648102 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-3e558524-2800-4612-93e3-dc799cb2e13d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc5cfe760daf4ea890b478f3ed031d0a'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:58.649107 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-3e558524-2800-4612-93e3-dc799cb2e13d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc5cfe760daf4ea890b478f3ed031d0a'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:58.657550 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-3e558524-2800-4612-93e3-dc799cb2e13d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc5cfe760daf4ea890b478f3ed031d0a'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:58.663075 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:58.672695 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-3e558524-2800-4612-93e3-dc799cb2e13d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc5cfe760daf4ea890b478f3ed031d0a'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:58.686451 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 896/1803] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:45:58 2022] DELETE /identity/v3/users/fc5cfe760daf4ea890b478f3ed031d0a => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:45:58.906725 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8467d84-22ae-4727-ba85-5cf8c20b52f4 None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:58.907217 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8467d84-22ae-4727-ba85-5cf8c20b52f4 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:58.907293 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8467d84-22ae-4727-ba85-5cf8c20b52f4 None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:58.908659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 908/1804] 10.222.0.20 () {54 vars in 921 bytes} [Wed Jul 6 13:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:45:58.916310 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0954758d-7870-4bd2-95e4-b6e8e37d5a16 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:58.916310 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0954758d-7870-4bd2-95e4-b6e8e37d5a16 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:58.916532 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0954758d-7870-4bd2-95e4-b6e8e37d5a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:58.937692 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-0954758d-7870-4bd2-95e4-b6e8e37d5a16 None None] MFA Rules not processed for user `890d2ae150d647a18dae25159cf97484`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:58.938891 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-0954758d-7870-4bd2-95e4-b6e8e37d5a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:58.986425 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8467d84-22ae-4727-ba85-5cf8c20b52f4 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:58.989957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-b8467d84-22ae-4727-ba85-5cf8c20b52f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:59.034890 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 897/1805] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Jul 6 13:45:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:45:59.053890 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0954758d-7870-4bd2-95e4-b6e8e37d5a16 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:59.056389 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-0954758d-7870-4bd2-95e4-b6e8e37d5a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:59.074603 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0954758d-7870-4bd2-95e4-b6e8e37d5a16 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:59.080852 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e9688-5aa3-4dba-8b20-92236af4d89d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '890d2ae150d647a18dae25159cf97484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:59.081673 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e9688-5aa3-4dba-8b20-92236af4d89d service placement] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:59.081962 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e9688-5aa3-4dba-8b20-92236af4d89d service placement] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:59.082235 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e9688-5aa3-4dba-8b20-92236af4d89d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:59.083355 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:59.094874 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e9688-5aa3-4dba-8b20-92236af4d89d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:59.097262 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e9688-5aa3-4dba-8b20-92236af4d89d service placement] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:59.126651 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8467d84-22ae-4727-ba85-5cf8c20b52f4 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:59.132018 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 898/1806] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:45:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:45:59.143440 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a682e0-fda9-4cc4-9d4b-18dd2ee2bac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:59.144717 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a682e0-fda9-4cc4-9d4b-18dd2ee2bac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:59.145276 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a682e0-fda9-4cc4-9d4b-18dd2ee2bac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:59.145669 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a682e0-fda9-4cc4-9d4b-18dd2ee2bac1 admin admin] PATH_INFO: `/v3/users/2d4da8cc84fe4716a316183a5a88096b` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:59.152183 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a682e0-fda9-4cc4-9d4b-18dd2ee2bac1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d4da8cc84fe4716a316183a5a88096b)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:59.154634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a682e0-fda9-4cc4-9d4b-18dd2ee2bac1 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:59.275082 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-54a682e0-fda9-4cc4-9d4b-18dd2ee2bac1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d4da8cc84fe4716a316183a5a88096b'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:59.302809 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-54a682e0-fda9-4cc4-9d4b-18dd2ee2bac1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d4da8cc84fe4716a316183a5a88096b'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:59.314581 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-54a682e0-fda9-4cc4-9d4b-18dd2ee2bac1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d4da8cc84fe4716a316183a5a88096b'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:59.315486 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-54a682e0-fda9-4cc4-9d4b-18dd2ee2bac1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d4da8cc84fe4716a316183a5a88096b'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:59.332717 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-54a682e0-fda9-4cc4-9d4b-18dd2ee2bac1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d4da8cc84fe4716a316183a5a88096b'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:59.341396 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 909/1807] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:45:58 2022] DELETE /identity/v3/users/2d4da8cc84fe4716a316183a5a88096b => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:45:59.405599 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e9688-5aa3-4dba-8b20-92236af4d89d service placement] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:59.409043 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-790e9688-5aa3-4dba-8b20-92236af4d89d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:59.426885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e9688-5aa3-4dba-8b20-92236af4d89d service placement] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:59.495234 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0db7b2-83d6-4a8d-88a6-ffb74f8dd14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:59.496246 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0db7b2-83d6-4a8d-88a6-ffb74f8dd14e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:59.496528 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0db7b2-83d6-4a8d-88a6-ffb74f8dd14e admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:59.496800 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0db7b2-83d6-4a8d-88a6-ffb74f8dd14e admin admin] PATH_INFO: `/v3/projects/17f971604c174b028821c9382de44729` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:59.498015 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0db7b2-83d6-4a8d-88a6-ffb74f8dd14e admin admin] RBAC: Authorizing `identity:delete_project(project_id=17f971604c174b028821c9382de44729)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:59.499913 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0db7b2-83d6-4a8d-88a6-ffb74f8dd14e admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:59.548386 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-8a0db7b2-83d6-4a8d-88a6-ffb74f8dd14e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17f971604c174b028821c9382de44729', 'request_id': 'req-8a0db7b2-83d6-4a8d-88a6-ffb74f8dd14e'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:59.556030 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-8a0db7b2-83d6-4a8d-88a6-ffb74f8dd14e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17f971604c174b028821c9382de44729', 'request_id': 'req-8a0db7b2-83d6-4a8d-88a6-ffb74f8dd14e'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:59.567959 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-8a0db7b2-83d6-4a8d-88a6-ffb74f8dd14e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17f971604c174b028821c9382de44729', 'request_id': 'req-8a0db7b2-83d6-4a8d-88a6-ffb74f8dd14e'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:59.575413 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:59.580922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-8a0db7b2-83d6-4a8d-88a6-ffb74f8dd14e admin admin] The token cache is being invalidate because project 17f971604c174b028821c9382de44729 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:45:59.581699 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-8a0db7b2-83d6-4a8d-88a6-ffb74f8dd14e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:59.582502 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-8a0db7b2-83d6-4a8d-88a6-ffb74f8dd14e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:59.584911 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 899/1808] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:45:59 2022] DELETE /identity/v3/projects/17f971604c174b028821c9382de44729 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:45:59.590605 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728485ce-085c-43af-a233-04dc8869b137 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:59.590901 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728485ce-085c-43af-a233-04dc8869b137 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:59.591184 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728485ce-085c-43af-a233-04dc8869b137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:59.610508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-728485ce-085c-43af-a233-04dc8869b137 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:59.611807 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-728485ce-085c-43af-a233-04dc8869b137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:59.619980 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0db7b2-83d6-4a8d-88a6-ffb74f8dd14e admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:59.621622 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-8a0db7b2-83d6-4a8d-88a6-ffb74f8dd14e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:59.675973 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 910/1809] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:59.683797 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a370226-20db-4600-9efc-b66abbf46402 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:59.684164 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a370226-20db-4600-9efc-b66abbf46402 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:59.684518 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a370226-20db-4600-9efc-b66abbf46402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:59.694099 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0db7b2-83d6-4a8d-88a6-ffb74f8dd14e admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:59.699726 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972686c4-3fb5-4f2b-8311-3f28e4c292c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:59.700716 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972686c4-3fb5-4f2b-8311-3f28e4c292c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:59.701084 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972686c4-3fb5-4f2b-8311-3f28e4c292c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:59.701367 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972686c4-3fb5-4f2b-8311-3f28e4c292c9 admin admin] PATH_INFO: `/v3/projects/40879f94775e43c982ff1a19c82db6fb` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:59.702694 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972686c4-3fb5-4f2b-8311-3f28e4c292c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40879f94775e43c982ff1a19c82db6fb)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:59.703310 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-5a370226-20db-4600-9efc-b66abbf46402 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:45:59.704566 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-5a370226-20db-4600-9efc-b66abbf46402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:59.704837 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972686c4-3fb5-4f2b-8311-3f28e4c292c9 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:59.713949 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 911/1810] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:45:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:45:59.718187 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a370226-20db-4600-9efc-b66abbf46402 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:59.728071 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a370226-20db-4600-9efc-b66abbf46402 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:59.734039 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90e358-4444-43a6-9040-16d6a1ccb59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:59.734813 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90e358-4444-43a6-9040-16d6a1ccb59b admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:59.735169 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90e358-4444-43a6-9040-16d6a1ccb59b admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:59.736065 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90e358-4444-43a6-9040-16d6a1ccb59b admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:59.736984 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf90e358-4444-43a6-9040-16d6a1ccb59b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:45:59.737329 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf90e358-4444-43a6-9040-16d6a1ccb59b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:59.739155 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf90e358-4444-43a6-9040-16d6a1ccb59b admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:59.752638 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 912/1811] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:45:59.756975 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90e358-4444-43a6-9040-16d6a1ccb59b admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:59.765090 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-972686c4-3fb5-4f2b-8311-3f28e4c292c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40879f94775e43c982ff1a19c82db6fb', 'request_id': 'req-972686c4-3fb5-4f2b-8311-3f28e4c292c9'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:59.768011 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90e358-4444-43a6-9040-16d6a1ccb59b admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:59.771970 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-972686c4-3fb5-4f2b-8311-3f28e4c292c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40879f94775e43c982ff1a19c82db6fb', 'request_id': 'req-972686c4-3fb5-4f2b-8311-3f28e4c292c9'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:59.781907 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-972686c4-3fb5-4f2b-8311-3f28e4c292c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40879f94775e43c982ff1a19c82db6fb', 'request_id': 'req-972686c4-3fb5-4f2b-8311-3f28e4c292c9'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:59.794172 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-972686c4-3fb5-4f2b-8311-3f28e4c292c9 admin admin] The token cache is being invalidate because project 40879f94775e43c982ff1a19c82db6fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:45:59.794482 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-972686c4-3fb5-4f2b-8311-3f28e4c292c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:59.795143 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-972686c4-3fb5-4f2b-8311-3f28e4c292c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:59.796969 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 900/1812] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:45:59 2022] DELETE /identity/v3/projects/40879f94775e43c982ff1a19c82db6fb => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:45:59.800639 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972686c4-3fb5-4f2b-8311-3f28e4c292c9 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:59.803219 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-972686c4-3fb5-4f2b-8311-3f28e4c292c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:59.817621 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:59.834453 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce24fe73-7404-4fff-ba3c-109fcd5b2975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:59.835464 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24fe73-7404-4fff-ba3c-109fcd5b2975 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:59.837546 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24fe73-7404-4fff-ba3c-109fcd5b2975 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:59.838239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24fe73-7404-4fff-ba3c-109fcd5b2975 admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:59.840250 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce24fe73-7404-4fff-ba3c-109fcd5b2975 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:59.844077 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce24fe73-7404-4fff-ba3c-109fcd5b2975 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:59.879685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972686c4-3fb5-4f2b-8311-3f28e4c292c9 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:59.885740 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53304d14-1b88-4145-928d-1b8dcd6ec571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:59.886684 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53304d14-1b88-4145-928d-1b8dcd6ec571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:59.887144 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53304d14-1b88-4145-928d-1b8dcd6ec571 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:59.887872 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53304d14-1b88-4145-928d-1b8dcd6ec571 admin admin] PATH_INFO: `/v3/users/ce5b1632fdb44e9dad331b1ed19d9b06` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:59.888838 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53304d14-1b88-4145-928d-1b8dcd6ec571 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce5b1632fdb44e9dad331b1ed19d9b06)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:59.890848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53304d14-1b88-4145-928d-1b8dcd6ec571 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:59.895430 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 913/1813] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:45:59 2022] POST /identity/v3/projects => generated 367 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:45:59.896207 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce24fe73-7404-4fff-ba3c-109fcd5b2975 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:45:59.899567 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-ce24fe73-7404-4fff-ba3c-109fcd5b2975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:45:59.940048 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:45:59.956755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-53304d14-1b88-4145-928d-1b8dcd6ec571 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce5b1632fdb44e9dad331b1ed19d9b06'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:59.966066 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-53304d14-1b88-4145-928d-1b8dcd6ec571 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce5b1632fdb44e9dad331b1ed19d9b06'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:59.966890 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-53304d14-1b88-4145-928d-1b8dcd6ec571 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce5b1632fdb44e9dad331b1ed19d9b06'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:59.973962 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-53304d14-1b88-4145-928d-1b8dcd6ec571 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce5b1632fdb44e9dad331b1ed19d9b06'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:59.974460 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce24fe73-7404-4fff-ba3c-109fcd5b2975 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:45:59.979547 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dbbea9-851d-411f-9d31-7932b11e1572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:45:59.980250 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbbea9-851d-411f-9d31-7932b11e1572 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:45:59.980537 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbbea9-851d-411f-9d31-7932b11e1572 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:45:59.980816 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbbea9-851d-411f-9d31-7932b11e1572 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:45:59.981554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dbbea9-851d-411f-9d31-7932b11e1572 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:45:59.983487 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dbbea9-851d-411f-9d31-7932b11e1572 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:45:59.986377 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-53304d14-1b88-4145-928d-1b8dcd6ec571 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce5b1632fdb44e9dad331b1ed19d9b06'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:45:59.992491 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 901/1814] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:45:59 2022] DELETE /identity/v3/users/ce5b1632fdb44e9dad331b1ed19d9b06 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:46:00.003173 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 914/1815] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:45:59 2022] POST /identity/v3/users => generated 284 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:46:00.007969 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53304d14-1b88-4145-928d-1b8dcd6ec571 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:00.009519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-53304d14-1b88-4145-928d-1b8dcd6ec571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:00.078250 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53304d14-1b88-4145-928d-1b8dcd6ec571 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:00.082942 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1ad88-6c24-4273-9349-5a143224c506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:00.083616 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1ad88-6c24-4273-9349-5a143224c506 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:00.083903 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1ad88-6c24-4273-9349-5a143224c506 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:00.084180 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1ad88-6c24-4273-9349-5a143224c506 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:00.084905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a1ad88-6c24-4273-9349-5a143224c506 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:00.086807 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a1ad88-6c24-4273-9349-5a143224c506 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:00.096949 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 902/1816] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:46:00.101153 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dbbea9-851d-411f-9d31-7932b11e1572 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:00.111443 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dbbea9-851d-411f-9d31-7932b11e1572 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:00.115955 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654cd90e-44fd-4725-b83a-a70ab4c314c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:00.117197 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cd90e-44fd-4725-b83a-a70ab4c314c9 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:00.117483 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cd90e-44fd-4725-b83a-a70ab4c314c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:00.117937 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cd90e-44fd-4725-b83a-a70ab4c314c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:00.118668 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654cd90e-44fd-4725-b83a-a70ab4c314c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:00.120248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654cd90e-44fd-4725-b83a-a70ab4c314c9 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:00.130709 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 915/1817] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:46:00.135615 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1ad88-6c24-4273-9349-5a143224c506 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:00.145649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1ad88-6c24-4273-9349-5a143224c506 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:00.151180 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6162ca-6caa-4fec-8997-e4a27d2ba25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:00.152134 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6162ca-6caa-4fec-8997-e4a27d2ba25c admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:00.152485 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6162ca-6caa-4fec-8997-e4a27d2ba25c admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:00.152831 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6162ca-6caa-4fec-8997-e4a27d2ba25c admin admin] PATH_INFO: `/v3/projects/89cafc4c24ad4d88aadd6e77e410bfe1/users/03a090f4e4f94dd2b5c9f50cef40b00f/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:00.171510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6162ca-6caa-4fec-8997-e4a27d2ba25c admin admin] RBAC: Authorizing `identity:create_grant(project_id=89cafc4c24ad4d88aadd6e77e410bfe1, user_id=03a090f4e4f94dd2b5c9f50cef40b00f, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:00.174360 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6162ca-6caa-4fec-8997-e4a27d2ba25c admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:00.178934 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654cd90e-44fd-4725-b83a-a70ab4c314c9 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:00.181425 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-654cd90e-44fd-4725-b83a-a70ab4c314c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:00.192757 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 903/1818] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:46:00 2022] PUT /identity/v3/projects/89cafc4c24ad4d88aadd6e77e410bfe1/users/03a090f4e4f94dd2b5c9f50cef40b00f/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:46:00.201510 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654cd90e-44fd-4725-b83a-a70ab4c314c9 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:00.206922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893b1b0-e69e-497f-948b-5326df96a1b9 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:00.207426 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893b1b0-e69e-497f-948b-5326df96a1b9 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:00.207885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893b1b0-e69e-497f-948b-5326df96a1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:00.237879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-c893b1b0-e69e-497f-948b-5326df96a1b9 None None] MFA Rules not processed for user `03a090f4e4f94dd2b5c9f50cef40b00f`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:00.239075 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-c893b1b0-e69e-497f-948b-5326df96a1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:00.261470 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3445d2-e7e7-4ffd-b367-5bfddcb5ea96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:00.262337 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3445d2-e7e7-4ffd-b367-5bfddcb5ea96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:00.262652 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3445d2-e7e7-4ffd-b367-5bfddcb5ea96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:00.262930 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3445d2-e7e7-4ffd-b367-5bfddcb5ea96 admin admin] PATH_INFO: `/v3/projects/9ff9e1b18a2747dbaf70558c24126667` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:00.264141 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3445d2-e7e7-4ffd-b367-5bfddcb5ea96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ff9e1b18a2747dbaf70558c24126667)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:00.266052 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3445d2-e7e7-4ffd-b367-5bfddcb5ea96 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:00.307112 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 904/1819] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:00 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:00.311033 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893b1b0-e69e-497f-948b-5326df96a1b9 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:00.321073 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8a3445d2-e7e7-4ffd-b367-5bfddcb5ea96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ff9e1b18a2747dbaf70558c24126667', 'request_id': 'req-8a3445d2-e7e7-4ffd-b367-5bfddcb5ea96'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:00.323343 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:00.323597 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893b1b0-e69e-497f-948b-5326df96a1b9 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:00.330371 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8a3445d2-e7e7-4ffd-b367-5bfddcb5ea96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ff9e1b18a2747dbaf70558c24126667', 'request_id': 'req-8a3445d2-e7e7-4ffd-b367-5bfddcb5ea96'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:00.330721 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:00.343054 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8a3445d2-e7e7-4ffd-b367-5bfddcb5ea96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ff9e1b18a2747dbaf70558c24126667', 'request_id': 'req-8a3445d2-e7e7-4ffd-b367-5bfddcb5ea96'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:00.360338 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8a3445d2-e7e7-4ffd-b367-5bfddcb5ea96 admin admin] The token cache is being invalidate because project 9ff9e1b18a2747dbaf70558c24126667 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:46:00.360536 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8a3445d2-e7e7-4ffd-b367-5bfddcb5ea96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:00.361357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8a3445d2-e7e7-4ffd-b367-5bfddcb5ea96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:00.366326 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 916/1820] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:46:00 2022] DELETE /identity/v3/projects/9ff9e1b18a2747dbaf70558c24126667 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:46:00.369370 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2edb7-7a9f-4d22-8d33-6e84e6e9f11b None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:00.369662 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2edb7-7a9f-4d22-8d33-6e84e6e9f11b None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:00.369945 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2edb7-7a9f-4d22-8d33-6e84e6e9f11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:00.392256 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-82a2edb7-7a9f-4d22-8d33-6e84e6e9f11b None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:00.394153 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-82a2edb7-7a9f-4d22-8d33-6e84e6e9f11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:00.406168 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815c1de-fa5f-45ee-bef0-e5ddbaa984e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:00.406855 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815c1de-fa5f-45ee-bef0-e5ddbaa984e5 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:00.407141 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815c1de-fa5f-45ee-bef0-e5ddbaa984e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:00.407418 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815c1de-fa5f-45ee-bef0-e5ddbaa984e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:00.408118 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815c1de-fa5f-45ee-bef0-e5ddbaa984e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:00.410063 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815c1de-fa5f-45ee-bef0-e5ddbaa984e5 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:00.441508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 905/1821] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:46:00 2022] POST /identity/v3/projects => generated 367 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:46:00.446573 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815c1de-fa5f-45ee-bef0-e5ddbaa984e5 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:00.448379 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-6815c1de-fa5f-45ee-bef0-e5ddbaa984e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:00.481134 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 917/1822] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:00.489618 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9473a9-4361-44cb-9718-6281d6f69d8d None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:00.489618 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9473a9-4361-44cb-9718-6281d6f69d8d None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:00.489936 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9473a9-4361-44cb-9718-6281d6f69d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:00.514927 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-0a9473a9-4361-44cb-9718-6281d6f69d8d None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:00.516185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-0a9473a9-4361-44cb-9718-6281d6f69d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:00.526208 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815c1de-fa5f-45ee-bef0-e5ddbaa984e5 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:00.528508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 918/1823] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:00.529785 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9473a9-4361-44cb-9718-6281d6f69d8d None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:00.532454 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8284b797-0712-47f4-8eb7-af133d5df2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:00.533246 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284b797-0712-47f4-8eb7-af133d5df2da admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:00.533598 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284b797-0712-47f4-8eb7-af133d5df2da admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:00.533949 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284b797-0712-47f4-8eb7-af133d5df2da admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:00.534853 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8284b797-0712-47f4-8eb7-af133d5df2da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:00.536907 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8284b797-0712-47f4-8eb7-af133d5df2da admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:00.540935 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9473a9-4361-44cb-9718-6281d6f69d8d None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:00.545772 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085ba50-3607-488c-99a8-35b93a054a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:00.546452 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085ba50-3607-488c-99a8-35b93a054a4d admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:00.547222 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085ba50-3607-488c-99a8-35b93a054a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:00.547554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085ba50-3607-488c-99a8-35b93a054a4d admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:00.548260 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085ba50-3607-488c-99a8-35b93a054a4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:46:00.548574 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085ba50-3607-488c-99a8-35b93a054a4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:00.550482 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085ba50-3607-488c-99a8-35b93a054a4d admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:00.564617 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 919/1824] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:46:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:46:00.570692 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085ba50-3607-488c-99a8-35b93a054a4d admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:00.574414 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 906/1825] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:46:00 2022] POST /identity/v3/users => generated 290 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:46:00.576550 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8284b797-0712-47f4-8eb7-af133d5df2da admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:00.581931 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085ba50-3607-488c-99a8-35b93a054a4d admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:00.587722 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7bba38-0929-4e8b-af77-f08600f0c963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:00.588523 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bba38-0929-4e8b-af77-f08600f0c963 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:00.588816 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bba38-0929-4e8b-af77-f08600f0c963 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:00.589122 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bba38-0929-4e8b-af77-f08600f0c963 admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:00.590106 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7bba38-0929-4e8b-af77-f08600f0c963 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:00.592084 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7bba38-0929-4e8b-af77-f08600f0c963 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:00.595095 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8284b797-0712-47f4-8eb7-af133d5df2da admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:00.602239 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf4c9e-e471-4893-b337-fd3bf34dec71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:00.603471 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf4c9e-e471-4893-b337-fd3bf34dec71 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:00.603471 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf4c9e-e471-4893-b337-fd3bf34dec71 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:00.603802 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf4c9e-e471-4893-b337-fd3bf34dec71 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:00.604802 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf4c9e-e471-4893-b337-fd3bf34dec71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:00.606729 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf4c9e-e471-4893-b337-fd3bf34dec71 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:00.624120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 907/1826] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:46:00.625024 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf4c9e-e471-4893-b337-fd3bf34dec71 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:00.636255 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf4c9e-e471-4893-b337-fd3bf34dec71 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:00.640692 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 920/1827] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:46:00 2022] POST /identity/v3/projects => generated 375 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:46:00.644526 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a54ae79-08cd-42bf-9ba3-b8a4bfc03335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:00.645509 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54ae79-08cd-42bf-9ba3-b8a4bfc03335 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:00.645798 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54ae79-08cd-42bf-9ba3-b8a4bfc03335 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:00.646102 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54ae79-08cd-42bf-9ba3-b8a4bfc03335 admin admin] PATH_INFO: `/v3/projects/1da84e46653e4b7a8fd27f412b707551/users/4be9e27c4a27402daf5411a140fe2624/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:00.646302 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7bba38-0929-4e8b-af77-f08600f0c963 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:00.664127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a54ae79-08cd-42bf-9ba3-b8a4bfc03335 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1da84e46653e4b7a8fd27f412b707551, user_id=4be9e27c4a27402daf5411a140fe2624, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:00.667518 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a54ae79-08cd-42bf-9ba3-b8a4bfc03335 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:00.684444 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 908/1828] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:46:00 2022] PUT /identity/v3/projects/1da84e46653e4b7a8fd27f412b707551/users/4be9e27c4a27402daf5411a140fe2624/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:00.688099 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a54ae79-08cd-42bf-9ba3-b8a4bfc03335 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:00.727063 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7bba38-0929-4e8b-af77-f08600f0c963 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:00.743387 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:00.770367 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a54ae79-08cd-42bf-9ba3-b8a4bfc03335 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:00.776173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d1067-0e08-4451-96a9-6b49218bb27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:00.776849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d1067-0e08-4451-96a9-6b49218bb27f admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:00.777203 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d1067-0e08-4451-96a9-6b49218bb27f admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:00.777483 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d1067-0e08-4451-96a9-6b49218bb27f admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:00.778212 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d1067-0e08-4451-96a9-6b49218bb27f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:00.779809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d1067-0e08-4451-96a9-6b49218bb27f admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:00.781057 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:00.790468 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 909/1829] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:46:00.794299 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d1067-0e08-4451-96a9-6b49218bb27f admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:00.795195 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab0044-47f6-41e6-82dc-d4b62d986cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:00.795867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab0044-47f6-41e6-82dc-d4b62d986cac admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:00.796150 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab0044-47f6-41e6-82dc-d4b62d986cac admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:00.796425 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab0044-47f6-41e6-82dc-d4b62d986cac admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:00.797140 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab0044-47f6-41e6-82dc-d4b62d986cac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:00.799040 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab0044-47f6-41e6-82dc-d4b62d986cac admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:00.804754 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d1067-0e08-4451-96a9-6b49218bb27f admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:00.809836 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ec083-8c8f-4673-88a3-4b81c37be014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:00.810871 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ec083-8c8f-4673-88a3-4b81c37be014 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:00.811185 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ec083-8c8f-4673-88a3-4b81c37be014 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:00.811495 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ec083-8c8f-4673-88a3-4b81c37be014 admin admin] PATH_INFO: `/v3/projects/1da84e46653e4b7a8fd27f412b707551/users/4be9e27c4a27402daf5411a140fe2624/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:00.813743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ec083-8c8f-4673-88a3-4b81c37be014 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1da84e46653e4b7a8fd27f412b707551, user_id=4be9e27c4a27402daf5411a140fe2624, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:00.816495 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ec083-8c8f-4673-88a3-4b81c37be014 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:00.821292 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 921/1830] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:46:00 2022] POST /identity/v3/users => generated 288 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:46:00.825235 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab0044-47f6-41e6-82dc-d4b62d986cac admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:00.828265 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 910/1831] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:46:00 2022] PUT /identity/v3/projects/1da84e46653e4b7a8fd27f412b707551/users/4be9e27c4a27402daf5411a140fe2624/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:46:00.835524 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab0044-47f6-41e6-82dc-d4b62d986cac admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:00.839563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ffc27-a8cc-43e7-861b-d274d12a6f1b None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:00.839874 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ffc27-a8cc-43e7-861b-d274d12a6f1b None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:00.840150 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ffc27-a8cc-43e7-861b-d274d12a6f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:00.868559 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-8c6ffc27-a8cc-43e7-861b-d274d12a6f1b None None] MFA Rules not processed for user `4be9e27c4a27402daf5411a140fe2624`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:00.869851 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-8c6ffc27-a8cc-43e7-861b-d274d12a6f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:00.892919 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed0eaa1-5c28-4387-9dd1-9224402b4310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:00.893621 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0eaa1-5c28-4387-9dd1-9224402b4310 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:00.893905 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0eaa1-5c28-4387-9dd1-9224402b4310 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:00.894183 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0eaa1-5c28-4387-9dd1-9224402b4310 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:00.894902 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed0eaa1-5c28-4387-9dd1-9224402b4310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:00.896483 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed0eaa1-5c28-4387-9dd1-9224402b4310 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:00.909269 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 922/1832] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:46:00.910539 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed0eaa1-5c28-4387-9dd1-9224402b4310 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:00.919876 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed0eaa1-5c28-4387-9dd1-9224402b4310 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:00.924394 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf8146-a68b-429f-8f2f-c3cfb303cb20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:00.925049 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf8146-a68b-429f-8f2f-c3cfb303cb20 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:00.925339 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf8146-a68b-429f-8f2f-c3cfb303cb20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:00.925612 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf8146-a68b-429f-8f2f-c3cfb303cb20 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:00.926572 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cf8146-a68b-429f-8f2f-c3cfb303cb20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:00.928112 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cf8146-a68b-429f-8f2f-c3cfb303cb20 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:00.941039 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 923/1833] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:46:00.942463 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf8146-a68b-429f-8f2f-c3cfb303cb20 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:00.948358 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 911/1834] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:00 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:46:00.953869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf8146-a68b-429f-8f2f-c3cfb303cb20 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:00.955819 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ffc27-a8cc-43e7-861b-d274d12a6f1b None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:00.957403 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-8c6ffc27-a8cc-43e7-861b-d274d12a6f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:00.960828 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161aa247-d2d8-411f-94a8-2eb8097c7d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:00.961867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161aa247-d2d8-411f-94a8-2eb8097c7d6c admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:00.962165 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161aa247-d2d8-411f-94a8-2eb8097c7d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:00.962452 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161aa247-d2d8-411f-94a8-2eb8097c7d6c admin admin] PATH_INFO: `/v3/projects/07ab486bbabf4869b22936fdca66a0c3/users/b42cf08a07d14ac1a800de7f0ca3517c/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:00.978933 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161aa247-d2d8-411f-94a8-2eb8097c7d6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=07ab486bbabf4869b22936fdca66a0c3, user_id=b42cf08a07d14ac1a800de7f0ca3517c, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:00.981913 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161aa247-d2d8-411f-94a8-2eb8097c7d6c admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:00.997219 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 924/1835] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:46:00 2022] PUT /identity/v3/projects/07ab486bbabf4869b22936fdca66a0c3/users/b42cf08a07d14ac1a800de7f0ca3517c/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:01.009271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b6593-77d5-4266-8da1-d27d2702acb1 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:01.009589 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b6593-77d5-4266-8da1-d27d2702acb1 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:01.009867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b6593-77d5-4266-8da1-d27d2702acb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:01.034822 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ffc27-a8cc-43e7-861b-d274d12a6f1b None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:01.038114 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-fd1b6593-77d5-4266-8da1-d27d2702acb1 None None] MFA Rules not processed for user `b42cf08a07d14ac1a800de7f0ca3517c`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:01.039434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-fd1b6593-77d5-4266-8da1-d27d2702acb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:01.090480 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357b6e3-e34a-43e2-9d24-be3b64a5a425 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:01.091048 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357b6e3-e34a-43e2-9d24-be3b64a5a425 service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:01.091333 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357b6e3-e34a-43e2-9d24-be3b64a5a425 service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:01.091603 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357b6e3-e34a-43e2-9d24-be3b64a5a425 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:01.122168 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 925/1836] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:01.122567 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:01.127515 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b6593-77d5-4266-8da1-d27d2702acb1 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:01.157621 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:01.164248 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1357b6e3-e34a-43e2-9d24-be3b64a5a425 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:01.165733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1357b6e3-e34a-43e2-9d24-be3b64a5a425 service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:01.175457 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 912/1837] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:46:00 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 13:46:01.192361 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b6593-77d5-4266-8da1-d27d2702acb1 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:01.198656 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c4d5e7-79e3-4976-8332-3d425f7e8e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:01.199470 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4d5e7-79e3-4976-8332-3d425f7e8e43 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:01.199798 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4d5e7-79e3-4976-8332-3d425f7e8e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:01.200141 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4d5e7-79e3-4976-8332-3d425f7e8e43 admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:01.200911 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c4d5e7-79e3-4976-8332-3d425f7e8e43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:01.203048 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c4d5e7-79e3-4976-8332-3d425f7e8e43 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:01.230882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 926/1838] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:46:01 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:46:01.235371 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357b6e3-e34a-43e2-9d24-be3b64a5a425 service cinder] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:01.247558 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357b6e3-e34a-43e2-9d24-be3b64a5a425 service cinder] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:01.259444 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c4d5e7-79e3-4976-8332-3d425f7e8e43 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:01.312112 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abe9a9f-2098-424d-b181-3f0a393b51e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:01.312797 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe9a9f-2098-424d-b181-3f0a393b51e5 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:01.313248 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe9a9f-2098-424d-b181-3f0a393b51e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:01.313532 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe9a9f-2098-424d-b181-3f0a393b51e5 admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:01.314273 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abe9a9f-2098-424d-b181-3f0a393b51e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:01.316189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c4d5e7-79e3-4976-8332-3d425f7e8e43 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:01.316295 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abe9a9f-2098-424d-b181-3f0a393b51e5 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:01.320739 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c69b9-d1ce-4f79-a8f8-00f7e93206b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:01.321477 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c69b9-d1ce-4f79-a8f8-00f7e93206b7 service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:01.321766 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c69b9-d1ce-4f79-a8f8-00f7e93206b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:01.322071 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c69b9-d1ce-4f79-a8f8-00f7e93206b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:01.323930 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-570c69b9-d1ce-4f79-a8f8-00f7e93206b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:01.337389 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 913/1839] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:46:01 2022] POST /identity/v3/users => generated 295 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:46:01.341268 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abe9a9f-2098-424d-b181-3f0a393b51e5 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:01.350787 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abe9a9f-2098-424d-b181-3f0a393b51e5 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:01.355402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daddbcff-28bb-40f1-88f1-5d7e9e12ce5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:01.356068 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daddbcff-28bb-40f1-88f1-5d7e9e12ce5b admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:01.356351 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daddbcff-28bb-40f1-88f1-5d7e9e12ce5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:01.356624 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daddbcff-28bb-40f1-88f1-5d7e9e12ce5b admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:01.357442 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daddbcff-28bb-40f1-88f1-5d7e9e12ce5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:01.359051 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daddbcff-28bb-40f1-88f1-5d7e9e12ce5b admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:01.369289 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 914/1840] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:46:01.373940 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daddbcff-28bb-40f1-88f1-5d7e9e12ce5b admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:01.385215 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daddbcff-28bb-40f1-88f1-5d7e9e12ce5b admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:01.389984 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed88793-b220-4967-b65e-e51c2a486297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:01.390879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed88793-b220-4967-b65e-e51c2a486297 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:01.391169 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed88793-b220-4967-b65e-e51c2a486297 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:01.391448 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed88793-b220-4967-b65e-e51c2a486297 admin admin] PATH_INFO: `/v3/projects/95cdac43499b49f49cc2e7dd4c7ebac8/users/41bdb637d9514efb893497c619e9a6c4/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:01.393575 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570c69b9-d1ce-4f79-a8f8-00f7e93206b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:01.395190 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570c69b9-d1ce-4f79-a8f8-00f7e93206b7 service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:01.407780 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 927/1841] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:46:01 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:46:01.408003 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed88793-b220-4967-b65e-e51c2a486297 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95cdac43499b49f49cc2e7dd4c7ebac8, user_id=41bdb637d9514efb893497c619e9a6c4, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:01.410606 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed88793-b220-4967-b65e-e51c2a486297 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:01.430210 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 915/1842] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:46:01 2022] PUT /identity/v3/projects/95cdac43499b49f49cc2e7dd4c7ebac8/users/41bdb637d9514efb893497c619e9a6c4/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:01.434518 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c69b9-d1ce-4f79-a8f8-00f7e93206b7 service cinder] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:01.445989 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c69b9-d1ce-4f79-a8f8-00f7e93206b7 service cinder] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:01.504017 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36adb8c-fb89-47fa-a72f-b0f6360ff8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:01.504725 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36adb8c-fb89-47fa-a72f-b0f6360ff8e5 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:01.505026 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36adb8c-fb89-47fa-a72f-b0f6360ff8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:01.505319 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36adb8c-fb89-47fa-a72f-b0f6360ff8e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:01.506054 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36adb8c-fb89-47fa-a72f-b0f6360ff8e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:01.508247 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36adb8c-fb89-47fa-a72f-b0f6360ff8e5 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:01.521345 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 928/1843] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:46:01.524184 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed88793-b220-4967-b65e-e51c2a486297 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:01.537680 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed88793-b220-4967-b65e-e51c2a486297 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:01.543280 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161e33e-5ef9-4ea9-aa7b-b808bf1bfa02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:01.544307 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161e33e-5ef9-4ea9-aa7b-b808bf1bfa02 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:01.544594 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161e33e-5ef9-4ea9-aa7b-b808bf1bfa02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:01.544904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161e33e-5ef9-4ea9-aa7b-b808bf1bfa02 admin admin] PATH_INFO: `/v3/projects/95cdac43499b49f49cc2e7dd4c7ebac8/users/41bdb637d9514efb893497c619e9a6c4/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:01.547049 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f161e33e-5ef9-4ea9-aa7b-b808bf1bfa02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95cdac43499b49f49cc2e7dd4c7ebac8, user_id=41bdb637d9514efb893497c619e9a6c4, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:01.549818 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f161e33e-5ef9-4ea9-aa7b-b808bf1bfa02 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:01.562211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 916/1844] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:46:01 2022] PUT /identity/v3/projects/95cdac43499b49f49cc2e7dd4c7ebac8/users/41bdb637d9514efb893497c619e9a6c4/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:01.573781 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5114d-0393-4e19-bce0-5eb16562b11f None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:01.574343 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5114d-0393-4e19-bce0-5eb16562b11f None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:01.574556 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5114d-0393-4e19-bce0-5eb16562b11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:01.581561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:01.756297 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-58b5114d-0393-4e19-bce0-5eb16562b11f None None] MFA Rules not processed for user `41bdb637d9514efb893497c619e9a6c4`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:01.758442 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-58b5114d-0393-4e19-bce0-5eb16562b11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:01.858866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 929/1845] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:01 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:01.871555 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161e33e-5ef9-4ea9-aa7b-b808bf1bfa02 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:01.930854 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161e33e-5ef9-4ea9-aa7b-b808bf1bfa02 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:01.934448 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b39273-5c9c-44aa-8171-f9b68c3be957 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:01.935214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b39273-5c9c-44aa-8171-f9b68c3be957 service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:01.935502 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b39273-5c9c-44aa-8171-f9b68c3be957 service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:01.935777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b39273-5c9c-44aa-8171-f9b68c3be957 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:01.953746 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:01.994421 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b39273-5c9c-44aa-8171-f9b68c3be957 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:01.995856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b39273-5c9c-44aa-8171-f9b68c3be957 service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:02.004736 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 917/1846] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:46:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:46:07.067607 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5114d-0393-4e19-bce0-5eb16562b11f None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:07.069201 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-58b5114d-0393-4e19-bce0-5eb16562b11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:07.080852 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5114d-0393-4e19-bce0-5eb16562b11f None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:07.086755 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305abb37-73a6-41f7-a46b-ce2af5c42a10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:07.087435 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305abb37-73a6-41f7-a46b-ce2af5c42a10 service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:07.087718 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305abb37-73a6-41f7-a46b-ce2af5c42a10 service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:07.088408 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305abb37-73a6-41f7-a46b-ce2af5c42a10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:07.090355 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-305abb37-73a6-41f7-a46b-ce2af5c42a10 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:07.158110 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305abb37-73a6-41f7-a46b-ce2af5c42a10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:07.159537 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305abb37-73a6-41f7-a46b-ce2af5c42a10 service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:07.171192 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 930/1847] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:46:07 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:46:10.775156 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b39273-5c9c-44aa-8171-f9b68c3be957 service cinder] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:10.783444 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b39273-5c9c-44aa-8171-f9b68c3be957 service cinder] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:10.786996 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0687cd-5208-469c-885f-c3412bafd95d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:10.787622 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0687cd-5208-469c-885f-c3412bafd95d service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:10.787918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0687cd-5208-469c-885f-c3412bafd95d service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:10.788346 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0687cd-5208-469c-885f-c3412bafd95d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:10.797141 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0687cd-5208-469c-885f-c3412bafd95d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:10.798595 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0687cd-5208-469c-885f-c3412bafd95d service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:10.808662 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 918/1848] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:46:10 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:46:12.590127 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305abb37-73a6-41f7-a46b-ce2af5c42a10 service cinder] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:12.659476 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305abb37-73a6-41f7-a46b-ce2af5c42a10 service cinder] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:12.664275 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d7c91-7393-4112-9753-99133e86094c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:12.665189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d7c91-7393-4112-9753-99133e86094c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:12.665476 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d7c91-7393-4112-9753-99133e86094c admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:12.665751 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d7c91-7393-4112-9753-99133e86094c admin admin] PATH_INFO: `/v3/users/03a090f4e4f94dd2b5c9f50cef40b00f` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:12.666958 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724d7c91-7393-4112-9753-99133e86094c admin admin] RBAC: Authorizing `identity:delete_user(user_id=03a090f4e4f94dd2b5c9f50cef40b00f)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:12.669191 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724d7c91-7393-4112-9753-99133e86094c admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:12.692929 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:12.720650 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-724d7c91-7393-4112-9753-99133e86094c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03a090f4e4f94dd2b5c9f50cef40b00f'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:12.728015 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-724d7c91-7393-4112-9753-99133e86094c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03a090f4e4f94dd2b5c9f50cef40b00f'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:12.737778 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-724d7c91-7393-4112-9753-99133e86094c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03a090f4e4f94dd2b5c9f50cef40b00f'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:12.738760 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-724d7c91-7393-4112-9753-99133e86094c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03a090f4e4f94dd2b5c9f50cef40b00f'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:12.758895 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-724d7c91-7393-4112-9753-99133e86094c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03a090f4e4f94dd2b5c9f50cef40b00f'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:12.765804 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 931/1849] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:46:12 2022] DELETE /identity/v3/users/03a090f4e4f94dd2b5c9f50cef40b00f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:12.770270 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee362e2-c19a-49da-9dd0-55584878ad86 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:12.770762 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee362e2-c19a-49da-9dd0-55584878ad86 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:12.770849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee362e2-c19a-49da-9dd0-55584878ad86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:12.791026 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-eee362e2-c19a-49da-9dd0-55584878ad86 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:12.792309 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-eee362e2-c19a-49da-9dd0-55584878ad86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:12.871157 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 919/1850] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:12.881228 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d7c91-7393-4112-9753-99133e86094c admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:12.883573 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-724d7c91-7393-4112-9753-99133e86094c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:12.952039 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d7c91-7393-4112-9753-99133e86094c admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:12.955688 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b905878-da32-439a-855a-87ee6b021290 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:12.956341 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b905878-da32-439a-855a-87ee6b021290 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:12.956628 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b905878-da32-439a-855a-87ee6b021290 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:12.956908 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b905878-da32-439a-855a-87ee6b021290 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:12.967065 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b905878-da32-439a-855a-87ee6b021290 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:12.968510 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b905878-da32-439a-855a-87ee6b021290 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:12.978289 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 932/1851] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:46:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:46:13.366840 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee362e2-c19a-49da-9dd0-55584878ad86 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:13.368438 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-eee362e2-c19a-49da-9dd0-55584878ad86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:13.382760 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee362e2-c19a-49da-9dd0-55584878ad86 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:13.387801 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12728f63-eced-40ac-88bf-70f9d6253944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:13.388682 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12728f63-eced-40ac-88bf-70f9d6253944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:13.389209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12728f63-eced-40ac-88bf-70f9d6253944 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:13.389497 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12728f63-eced-40ac-88bf-70f9d6253944 admin admin] PATH_INFO: `/v3/projects/89cafc4c24ad4d88aadd6e77e410bfe1` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:13.390781 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12728f63-eced-40ac-88bf-70f9d6253944 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89cafc4c24ad4d88aadd6e77e410bfe1)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:13.392705 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12728f63-eced-40ac-88bf-70f9d6253944 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:13.439844 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-12728f63-eced-40ac-88bf-70f9d6253944 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89cafc4c24ad4d88aadd6e77e410bfe1', 'request_id': 'req-12728f63-eced-40ac-88bf-70f9d6253944'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:13.446429 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-12728f63-eced-40ac-88bf-70f9d6253944 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89cafc4c24ad4d88aadd6e77e410bfe1', 'request_id': 'req-12728f63-eced-40ac-88bf-70f9d6253944'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:13.453574 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-12728f63-eced-40ac-88bf-70f9d6253944 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89cafc4c24ad4d88aadd6e77e410bfe1', 'request_id': 'req-12728f63-eced-40ac-88bf-70f9d6253944'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:13.464642 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-12728f63-eced-40ac-88bf-70f9d6253944 admin admin] The token cache is being invalidate because project 89cafc4c24ad4d88aadd6e77e410bfe1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:46:13.464935 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-12728f63-eced-40ac-88bf-70f9d6253944 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:13.465602 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-12728f63-eced-40ac-88bf-70f9d6253944 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:13.467149 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 920/1852] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:46:13 2022] DELETE /identity/v3/projects/89cafc4c24ad4d88aadd6e77e410bfe1 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:13.471075 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b905878-da32-439a-855a-87ee6b021290 service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:13.481248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b905878-da32-439a-855a-87ee6b021290 service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:13.537166 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72269303-dc1d-4935-92f1-fb8248ba2229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:13.538073 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72269303-dc1d-4935-92f1-fb8248ba2229 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:13.538353 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72269303-dc1d-4935-92f1-fb8248ba2229 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:13.538627 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72269303-dc1d-4935-92f1-fb8248ba2229 admin admin] PATH_INFO: `/v3/users/4be9e27c4a27402daf5411a140fe2624` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:13.539794 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72269303-dc1d-4935-92f1-fb8248ba2229 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4be9e27c4a27402daf5411a140fe2624)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:13.541718 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72269303-dc1d-4935-92f1-fb8248ba2229 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:13.585478 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:13.587302 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-72269303-dc1d-4935-92f1-fb8248ba2229 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4be9e27c4a27402daf5411a140fe2624'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:13.594673 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-72269303-dc1d-4935-92f1-fb8248ba2229 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4be9e27c4a27402daf5411a140fe2624'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:13.602527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-72269303-dc1d-4935-92f1-fb8248ba2229 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4be9e27c4a27402daf5411a140fe2624'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:13.603177 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-72269303-dc1d-4935-92f1-fb8248ba2229 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4be9e27c4a27402daf5411a140fe2624'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:13.614612 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-72269303-dc1d-4935-92f1-fb8248ba2229 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4be9e27c4a27402daf5411a140fe2624'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:13.620406 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 933/1853] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:46:13 2022] DELETE /identity/v3/users/4be9e27c4a27402daf5411a140fe2624 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:13.985764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12728f63-eced-40ac-88bf-70f9d6253944 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:13.987364 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-12728f63-eced-40ac-88bf-70f9d6253944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:14.003741 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12728f63-eced-40ac-88bf-70f9d6253944 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:14.044261 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:14.079394 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cdc573-880f-44c2-8177-4a38f2ef2494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:14.080417 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdc573-880f-44c2-8177-4a38f2ef2494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:14.080776 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdc573-880f-44c2-8177-4a38f2ef2494 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:14.081554 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdc573-880f-44c2-8177-4a38f2ef2494 admin admin] PATH_INFO: `/v3/projects/1da84e46653e4b7a8fd27f412b707551` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:14.082286 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cdc573-880f-44c2-8177-4a38f2ef2494 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1da84e46653e4b7a8fd27f412b707551)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:14.084218 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cdc573-880f-44c2-8177-4a38f2ef2494 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:14.134873 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-72cdc573-880f-44c2-8177-4a38f2ef2494 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1da84e46653e4b7a8fd27f412b707551', 'request_id': 'req-72cdc573-880f-44c2-8177-4a38f2ef2494'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:14.141702 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-72cdc573-880f-44c2-8177-4a38f2ef2494 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1da84e46653e4b7a8fd27f412b707551', 'request_id': 'req-72cdc573-880f-44c2-8177-4a38f2ef2494'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:14.151144 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-72cdc573-880f-44c2-8177-4a38f2ef2494 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1da84e46653e4b7a8fd27f412b707551', 'request_id': 'req-72cdc573-880f-44c2-8177-4a38f2ef2494'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:14.167953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-72cdc573-880f-44c2-8177-4a38f2ef2494 admin admin] The token cache is being invalidate because project 1da84e46653e4b7a8fd27f412b707551 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:46:14.168414 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-72cdc573-880f-44c2-8177-4a38f2ef2494 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:14.169384 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-72cdc573-880f-44c2-8177-4a38f2ef2494 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:14.171821 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 921/1854] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:46:13 2022] DELETE /identity/v3/projects/1da84e46653e4b7a8fd27f412b707551 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:14.177505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47853f8-099d-4f43-ba46-eee643232e65 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:14.177965 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47853f8-099d-4f43-ba46-eee643232e65 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:14.178382 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47853f8-099d-4f43-ba46-eee643232e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:14.207359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-d47853f8-099d-4f43-ba46-eee643232e65 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:14.208746 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-d47853f8-099d-4f43-ba46-eee643232e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:14.328245 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 934/1855] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:46:14.332982 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f2c0a-e7da-4800-ae49-8750172b7dfb None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:14.333243 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f2c0a-e7da-4800-ae49-8750172b7dfb None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:14.333695 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f2c0a-e7da-4800-ae49-8750172b7dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:14.355673 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-0a8f2c0a-e7da-4800-ae49-8750172b7dfb None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:14.356874 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-0a8f2c0a-e7da-4800-ae49-8750172b7dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:14.365870 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 922/1856] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:14.370136 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47853f8-099d-4f43-ba46-eee643232e65 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:14.379755 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47853f8-099d-4f43-ba46-eee643232e65 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:14.385181 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b2168-0836-4cfa-b689-82b4e01b6f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:14.385726 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b2168-0836-4cfa-b689-82b4e01b6f25 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:14.386011 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b2168-0836-4cfa-b689-82b4e01b6f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:14.386289 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b2168-0836-4cfa-b689-82b4e01b6f25 admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:14.387058 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47b2168-0836-4cfa-b689-82b4e01b6f25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:46:14.387369 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47b2168-0836-4cfa-b689-82b4e01b6f25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:14.388950 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47b2168-0836-4cfa-b689-82b4e01b6f25 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:14.401289 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 935/1857] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:46:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:46:14.405664 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f2c0a-e7da-4800-ae49-8750172b7dfb None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:14.415528 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f2c0a-e7da-4800-ae49-8750172b7dfb None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:14.420153 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a99dba1-049a-47eb-b719-fdc44218152a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:14.420801 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99dba1-049a-47eb-b719-fdc44218152a admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:14.421121 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99dba1-049a-47eb-b719-fdc44218152a admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:14.421400 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99dba1-049a-47eb-b719-fdc44218152a admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:14.422083 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a99dba1-049a-47eb-b719-fdc44218152a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:14.423970 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a99dba1-049a-47eb-b719-fdc44218152a admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:14.452529 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 923/1858] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:46:14 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:46:14.453693 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b2168-0836-4cfa-b689-82b4e01b6f25 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:14.464191 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b2168-0836-4cfa-b689-82b4e01b6f25 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:14.524249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2c927-bd5a-41b7-a34c-5c9935bf2169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:14.524929 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2c927-bd5a-41b7-a34c-5c9935bf2169 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:14.525244 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2c927-bd5a-41b7-a34c-5c9935bf2169 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:14.525525 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2c927-bd5a-41b7-a34c-5c9935bf2169 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:14.526222 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2c927-bd5a-41b7-a34c-5c9935bf2169 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:14.528108 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2c927-bd5a-41b7-a34c-5c9935bf2169 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:14.548067 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 936/1859] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:46:14 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:46:14.552370 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a99dba1-049a-47eb-b719-fdc44218152a admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:14.563133 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a99dba1-049a-47eb-b719-fdc44218152a admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:14.567697 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83cc02-7c71-4c19-8f65-3c75cdf6c38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:14.568352 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83cc02-7c71-4c19-8f65-3c75cdf6c38b admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:14.568635 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83cc02-7c71-4c19-8f65-3c75cdf6c38b admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:14.568925 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83cc02-7c71-4c19-8f65-3c75cdf6c38b admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:14.569821 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df83cc02-7c71-4c19-8f65-3c75cdf6c38b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:14.571410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df83cc02-7c71-4c19-8f65-3c75cdf6c38b admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:14.581832 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 924/1860] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:46:14.586459 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2c927-bd5a-41b7-a34c-5c9935bf2169 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:14.596747 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2c927-bd5a-41b7-a34c-5c9935bf2169 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:14.602468 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901de63-b038-439e-a903-7fc1d6294606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:14.603165 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901de63-b038-439e-a903-7fc1d6294606 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:14.603452 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901de63-b038-439e-a903-7fc1d6294606 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:14.603731 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901de63-b038-439e-a903-7fc1d6294606 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:14.604463 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c901de63-b038-439e-a903-7fc1d6294606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:14.606454 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c901de63-b038-439e-a903-7fc1d6294606 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:14.616706 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 937/1861] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:46:14.621011 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83cc02-7c71-4c19-8f65-3c75cdf6c38b admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:14.631885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83cc02-7c71-4c19-8f65-3c75cdf6c38b admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:14.636657 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89af493-b495-4770-adf1-949fe684bd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:14.637585 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89af493-b495-4770-adf1-949fe684bd2e admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:14.637871 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89af493-b495-4770-adf1-949fe684bd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:14.638147 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89af493-b495-4770-adf1-949fe684bd2e admin admin] PATH_INFO: `/v3/projects/c7a3d09eab8347a29c3cca1f3fcafac8/users/555f6a10bbe149f699fafd7f144609e9/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:14.653448 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89af493-b495-4770-adf1-949fe684bd2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7a3d09eab8347a29c3cca1f3fcafac8, user_id=555f6a10bbe149f699fafd7f144609e9, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:14.656142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89af493-b495-4770-adf1-949fe684bd2e admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:14.670108 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 925/1862] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:46:14 2022] PUT /identity/v3/projects/c7a3d09eab8347a29c3cca1f3fcafac8/users/555f6a10bbe149f699fafd7f144609e9/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:14.681869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ceb22-d029-41c2-8e1d-b8bb4b48c5b0 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:14.682182 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ceb22-d029-41c2-8e1d-b8bb4b48c5b0 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:14.682465 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ceb22-d029-41c2-8e1d-b8bb4b48c5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:14.708892 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-527ceb22-d029-41c2-8e1d-b8bb4b48c5b0 None None] MFA Rules not processed for user `555f6a10bbe149f699fafd7f144609e9`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:14.710755 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-527ceb22-d029-41c2-8e1d-b8bb4b48c5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:14.727862 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:14.776198 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 938/1863] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:14.780725 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89af493-b495-4770-adf1-949fe684bd2e admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:14.838047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:14.844058 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89af493-b495-4770-adf1-949fe684bd2e admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:14.849880 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557984a-e6c6-47c0-9f3e-8e805dde4a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:14.850615 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557984a-e6c6-47c0-9f3e-8e805dde4a89 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:14.850898 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557984a-e6c6-47c0-9f3e-8e805dde4a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:14.851172 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557984a-e6c6-47c0-9f3e-8e805dde4a89 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:14.851888 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b557984a-e6c6-47c0-9f3e-8e805dde4a89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:14.853878 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b557984a-e6c6-47c0-9f3e-8e805dde4a89 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:14.878255 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 926/1864] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:46:14 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:46:14.883046 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ceb22-d029-41c2-8e1d-b8bb4b48c5b0 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:14.894088 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ceb22-d029-41c2-8e1d-b8bb4b48c5b0 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:14.951556 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f5120-72ae-4ebf-8e94-00d8822cb64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:14.952230 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f5120-72ae-4ebf-8e94-00d8822cb64d admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:14.952515 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f5120-72ae-4ebf-8e94-00d8822cb64d admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:14.952796 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f5120-72ae-4ebf-8e94-00d8822cb64d admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:14.953540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71f5120-72ae-4ebf-8e94-00d8822cb64d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:14.955428 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71f5120-72ae-4ebf-8e94-00d8822cb64d admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:14.977039 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 939/1865] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:46:14 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:46:14.979057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557984a-e6c6-47c0-9f3e-8e805dde4a89 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:14.989832 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557984a-e6c6-47c0-9f3e-8e805dde4a89 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:14.994467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eec5f5-4f67-420a-afb0-4cf27d533cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:14.995126 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eec5f5-4f67-420a-afb0-4cf27d533cbe admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:14.995411 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eec5f5-4f67-420a-afb0-4cf27d533cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:14.995686 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eec5f5-4f67-420a-afb0-4cf27d533cbe admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:14.996402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eec5f5-4f67-420a-afb0-4cf27d533cbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:14.998010 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eec5f5-4f67-420a-afb0-4cf27d533cbe admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:15.009079 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 927/1866] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:46:15.013441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f5120-72ae-4ebf-8e94-00d8822cb64d admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:15.024959 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f5120-72ae-4ebf-8e94-00d8822cb64d admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:15.029568 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14341aba-cf2b-4e06-8429-fc9d0f0680bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:15.030492 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14341aba-cf2b-4e06-8429-fc9d0f0680bc admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:15.030773 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14341aba-cf2b-4e06-8429-fc9d0f0680bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:15.031045 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14341aba-cf2b-4e06-8429-fc9d0f0680bc admin admin] PATH_INFO: `/v3/projects/66750fbf7aa6413cbc956a5613d6b031/users/155b481a683b448b989286cff41c4758/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:15.048195 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14341aba-cf2b-4e06-8429-fc9d0f0680bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=66750fbf7aa6413cbc956a5613d6b031, user_id=155b481a683b448b989286cff41c4758, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:15.050893 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14341aba-cf2b-4e06-8429-fc9d0f0680bc admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:15.066875 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 940/1867] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:46:15 2022] PUT /identity/v3/projects/66750fbf7aa6413cbc956a5613d6b031/users/155b481a683b448b989286cff41c4758/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:15.071481 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eec5f5-4f67-420a-afb0-4cf27d533cbe admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:15.082242 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eec5f5-4f67-420a-afb0-4cf27d533cbe admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:15.146167 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ae3a0-42a0-42f3-bf1d-d0d17668d40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:15.146848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ae3a0-42a0-42f3-bf1d-d0d17668d40d admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:15.147120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ae3a0-42a0-42f3-bf1d-d0d17668d40d admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:15.147440 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ae3a0-42a0-42f3-bf1d-d0d17668d40d admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:15.148536 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530ae3a0-42a0-42f3-bf1d-d0d17668d40d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:15.150192 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530ae3a0-42a0-42f3-bf1d-d0d17668d40d admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:15.163462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 928/1868] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:46:15 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:46:15.164841 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14341aba-cf2b-4e06-8429-fc9d0f0680bc admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:15.175076 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14341aba-cf2b-4e06-8429-fc9d0f0680bc admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:15.180673 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9a4a2-df2c-422e-9afa-562018a2f145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:15.181581 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9a4a2-df2c-422e-9afa-562018a2f145 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:15.181863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9a4a2-df2c-422e-9afa-562018a2f145 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:15.182135 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9a4a2-df2c-422e-9afa-562018a2f145 admin admin] PATH_INFO: `/v3/projects/66750fbf7aa6413cbc956a5613d6b031/users/155b481a683b448b989286cff41c4758/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:15.184080 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd9a4a2-df2c-422e-9afa-562018a2f145 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66750fbf7aa6413cbc956a5613d6b031, user_id=155b481a683b448b989286cff41c4758, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:15.186838 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd9a4a2-df2c-422e-9afa-562018a2f145 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:15.198165 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 941/1869] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:46:15 2022] PUT /identity/v3/projects/66750fbf7aa6413cbc956a5613d6b031/users/155b481a683b448b989286cff41c4758/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:15.208339 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1768b0-43df-46ee-9536-0811fea95811 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:15.208635 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1768b0-43df-46ee-9536-0811fea95811 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:15.208919 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1768b0-43df-46ee-9536-0811fea95811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:15.235832 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-1b1768b0-43df-46ee-9536-0811fea95811 None None] MFA Rules not processed for user `155b481a683b448b989286cff41c4758`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:15.237092 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-1b1768b0-43df-46ee-9536-0811fea95811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:15.309328 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 929/1870] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:15 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:15.320363 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9a4a2-df2c-422e-9afa-562018a2f145 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:15.322066 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-8fd9a4a2-df2c-422e-9afa-562018a2f145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:15.391844 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9a4a2-df2c-422e-9afa-562018a2f145 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:15.395755 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12b992-dcc5-457c-b7d0-483902d7dc70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:15.396408 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12b992-dcc5-457c-b7d0-483902d7dc70 service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:15.396689 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12b992-dcc5-457c-b7d0-483902d7dc70 service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:15.396964 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12b992-dcc5-457c-b7d0-483902d7dc70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:15.427865 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:15.471532 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb12b992-dcc5-457c-b7d0-483902d7dc70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:15.473194 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb12b992-dcc5-457c-b7d0-483902d7dc70 service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:15.489143 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 942/1871] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:46:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:46:18.416406 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e203637-5226-494c-a7ef-c60e747e2b99 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:18.416920 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e203637-5226-494c-a7ef-c60e747e2b99 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:18.417155 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e203637-5226-494c-a7ef-c60e747e2b99 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:18.418681 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 930/1872] 10.222.0.20 () {54 vars in 921 bytes} [Wed Jul 6 13:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:46:18.425071 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57ac3b-7aac-42e3-8199-53e3c7cc5a46 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:18.425528 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57ac3b-7aac-42e3-8199-53e3c7cc5a46 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:18.425706 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57ac3b-7aac-42e3-8199-53e3c7cc5a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:18.445731 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-fc57ac3b-7aac-42e3-8199-53e3c7cc5a46 None None] MFA Rules not processed for user `6647eb1e8a1544ef9d3a9806f2d1fc51`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:18.446940 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-fc57ac3b-7aac-42e3-8199-53e3c7cc5a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:18.526206 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 943/1873] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Jul 6 13:46:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:18.550244 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e203637-5226-494c-a7ef-c60e747e2b99 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:18.552376 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-1e203637-5226-494c-a7ef-c60e747e2b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:18.574391 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e203637-5226-494c-a7ef-c60e747e2b99 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:18.584217 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749142a-6cd4-4395-827b-a74046d78018 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:18.586277 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749142a-6cd4-4395-827b-a74046d78018 service nova] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:18.586766 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749142a-6cd4-4395-827b-a74046d78018 service nova] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:18.587124 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749142a-6cd4-4395-827b-a74046d78018 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:18.598693 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749142a-6cd4-4395-827b-a74046d78018 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:18.603422 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749142a-6cd4-4395-827b-a74046d78018 service nova] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:18.620287 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 931/1874] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:46:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:46:19.167217 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57ac3b-7aac-42e3-8199-53e3c7cc5a46 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:19.231869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57ac3b-7aac-42e3-8199-53e3c7cc5a46 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:19.236406 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ad8d9-5969-4da6-a385-11bf9a67febb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:19.237314 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ad8d9-5969-4da6-a385-11bf9a67febb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:19.237599 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ad8d9-5969-4da6-a385-11bf9a67febb admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:19.237878 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ad8d9-5969-4da6-a385-11bf9a67febb admin admin] PATH_INFO: `/v3/users/555f6a10bbe149f699fafd7f144609e9` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:19.239069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ad8d9-5969-4da6-a385-11bf9a67febb admin admin] RBAC: Authorizing `identity:delete_user(user_id=555f6a10bbe149f699fafd7f144609e9)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:19.241121 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ad8d9-5969-4da6-a385-11bf9a67febb admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:19.288343 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-7c8ad8d9-5969-4da6-a385-11bf9a67febb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '555f6a10bbe149f699fafd7f144609e9'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:19.297556 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-7c8ad8d9-5969-4da6-a385-11bf9a67febb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '555f6a10bbe149f699fafd7f144609e9'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:19.302690 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:19.306153 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-7c8ad8d9-5969-4da6-a385-11bf9a67febb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '555f6a10bbe149f699fafd7f144609e9'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:19.307389 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-7c8ad8d9-5969-4da6-a385-11bf9a67febb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '555f6a10bbe149f699fafd7f144609e9'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:19.321167 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-7c8ad8d9-5969-4da6-a385-11bf9a67febb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '555f6a10bbe149f699fafd7f144609e9'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:19.327134 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 944/1875] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:46:19 2022] DELETE /identity/v3/users/555f6a10bbe149f699fafd7f144609e9 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:19.331512 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a3bcb-a522-4ea9-a488-c2a7fd096ac5 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:19.331802 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a3bcb-a522-4ea9-a488-c2a7fd096ac5 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:19.332081 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a3bcb-a522-4ea9-a488-c2a7fd096ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:19.349984 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-0e9a3bcb-a522-4ea9-a488-c2a7fd096ac5 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:19.351415 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-0e9a3bcb-a522-4ea9-a488-c2a7fd096ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:19.383276 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:19.411262 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 932/1876] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:19.420882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ad8d9-5969-4da6-a385-11bf9a67febb admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:19.422313 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-7c8ad8d9-5969-4da6-a385-11bf9a67febb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:19.477453 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ad8d9-5969-4da6-a385-11bf9a67febb admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:19.481145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5c074-5aa8-44ee-9674-f51a13bee8d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:19.481782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5c074-5aa8-44ee-9674-f51a13bee8d6 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:19.482067 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5c074-5aa8-44ee-9674-f51a13bee8d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:19.482341 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5c074-5aa8-44ee-9674-f51a13bee8d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:19.491286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce5c074-5aa8-44ee-9674-f51a13bee8d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:19.492722 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce5c074-5aa8-44ee-9674-f51a13bee8d6 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:19.502143 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 945/1877] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:46:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:46:19.770433 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a3bcb-a522-4ea9-a488-c2a7fd096ac5 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:19.772846 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-0e9a3bcb-a522-4ea9-a488-c2a7fd096ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:19.787972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a3bcb-a522-4ea9-a488-c2a7fd096ac5 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:19.794762 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868d24f-2969-4c8b-b5d3-958bb37756f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:19.795659 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868d24f-2969-4c8b-b5d3-958bb37756f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:19.795905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868d24f-2969-4c8b-b5d3-958bb37756f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:19.796259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868d24f-2969-4c8b-b5d3-958bb37756f5 admin admin] PATH_INFO: `/v3/projects/c7a3d09eab8347a29c3cca1f3fcafac8` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:19.798435 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8868d24f-2969-4c8b-b5d3-958bb37756f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7a3d09eab8347a29c3cca1f3fcafac8)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:19.800536 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8868d24f-2969-4c8b-b5d3-958bb37756f5 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:19.860665 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-8868d24f-2969-4c8b-b5d3-958bb37756f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7a3d09eab8347a29c3cca1f3fcafac8', 'request_id': 'req-8868d24f-2969-4c8b-b5d3-958bb37756f5'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:19.871871 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-8868d24f-2969-4c8b-b5d3-958bb37756f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7a3d09eab8347a29c3cca1f3fcafac8', 'request_id': 'req-8868d24f-2969-4c8b-b5d3-958bb37756f5'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:19.885290 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-8868d24f-2969-4c8b-b5d3-958bb37756f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7a3d09eab8347a29c3cca1f3fcafac8', 'request_id': 'req-8868d24f-2969-4c8b-b5d3-958bb37756f5'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:19.897728 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-8868d24f-2969-4c8b-b5d3-958bb37756f5 admin admin] The token cache is being invalidate because project c7a3d09eab8347a29c3cca1f3fcafac8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:46:19.898010 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-8868d24f-2969-4c8b-b5d3-958bb37756f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:19.898656 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-8868d24f-2969-4c8b-b5d3-958bb37756f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:19.900301 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 933/1878] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:46:19 2022] DELETE /identity/v3/projects/c7a3d09eab8347a29c3cca1f3fcafac8 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:19.904281 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5c074-5aa8-44ee-9674-f51a13bee8d6 service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:19.914582 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5c074-5aa8-44ee-9674-f51a13bee8d6 service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:19.983485 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6856a7a-467c-40ca-83db-8ad85cf59f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:19.984399 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6856a7a-467c-40ca-83db-8ad85cf59f51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:19.984687 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6856a7a-467c-40ca-83db-8ad85cf59f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:19.984967 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6856a7a-467c-40ca-83db-8ad85cf59f51 admin admin] PATH_INFO: `/v3/users/155b481a683b448b989286cff41c4758` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:19.986194 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6856a7a-467c-40ca-83db-8ad85cf59f51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=155b481a683b448b989286cff41c4758)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:19.988097 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6856a7a-467c-40ca-83db-8ad85cf59f51 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:20.039032 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-d6856a7a-467c-40ca-83db-8ad85cf59f51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '155b481a683b448b989286cff41c4758'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:20.047130 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-d6856a7a-467c-40ca-83db-8ad85cf59f51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '155b481a683b448b989286cff41c4758'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:20.054993 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-d6856a7a-467c-40ca-83db-8ad85cf59f51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '155b481a683b448b989286cff41c4758'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:20.055674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-d6856a7a-467c-40ca-83db-8ad85cf59f51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '155b481a683b448b989286cff41c4758'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:20.067596 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-d6856a7a-467c-40ca-83db-8ad85cf59f51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '155b481a683b448b989286cff41c4758'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:20.070408 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:20.079717 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 946/1879] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:46:19 2022] DELETE /identity/v3/users/155b481a683b448b989286cff41c4758 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:20.453646 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868d24f-2969-4c8b-b5d3-958bb37756f5 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:20.455168 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-8868d24f-2969-4c8b-b5d3-958bb37756f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:20.468395 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868d24f-2969-4c8b-b5d3-958bb37756f5 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:20.531418 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f4f638-6f10-454d-87a2-89b5f2e19c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:20.532286 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4f638-6f10-454d-87a2-89b5f2e19c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:20.532569 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4f638-6f10-454d-87a2-89b5f2e19c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:20.532848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4f638-6f10-454d-87a2-89b5f2e19c6f admin admin] PATH_INFO: `/v3/projects/66750fbf7aa6413cbc956a5613d6b031` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:20.534045 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f4f638-6f10-454d-87a2-89b5f2e19c6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=66750fbf7aa6413cbc956a5613d6b031)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:20.535943 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f4f638-6f10-454d-87a2-89b5f2e19c6f admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:20.582003 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-92f4f638-6f10-454d-87a2-89b5f2e19c6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66750fbf7aa6413cbc956a5613d6b031', 'request_id': 'req-92f4f638-6f10-454d-87a2-89b5f2e19c6f'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:20.587875 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-92f4f638-6f10-454d-87a2-89b5f2e19c6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66750fbf7aa6413cbc956a5613d6b031', 'request_id': 'req-92f4f638-6f10-454d-87a2-89b5f2e19c6f'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:20.595990 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-92f4f638-6f10-454d-87a2-89b5f2e19c6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66750fbf7aa6413cbc956a5613d6b031', 'request_id': 'req-92f4f638-6f10-454d-87a2-89b5f2e19c6f'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:20.599062 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:20.609365 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-92f4f638-6f10-454d-87a2-89b5f2e19c6f admin admin] The token cache is being invalidate because project 66750fbf7aa6413cbc956a5613d6b031 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:46:20.609863 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-92f4f638-6f10-454d-87a2-89b5f2e19c6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:20.610708 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-92f4f638-6f10-454d-87a2-89b5f2e19c6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:20.612269 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 934/1880] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:46:20 2022] DELETE /identity/v3/projects/66750fbf7aa6413cbc956a5613d6b031 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:20.617995 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92576c38-9c4b-48d3-b904-b04f9e63ee7f None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:20.618474 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92576c38-9c4b-48d3-b904-b04f9e63ee7f None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:20.618597 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92576c38-9c4b-48d3-b904-b04f9e63ee7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:20.636296 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-92576c38-9c4b-48d3-b904-b04f9e63ee7f None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:20.637507 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-92576c38-9c4b-48d3-b904-b04f9e63ee7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:20.695606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 947/1881] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:20.703124 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816628cf-c5b5-4503-ad2e-4e15d9b9e076 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:20.703416 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816628cf-c5b5-4503-ad2e-4e15d9b9e076 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:20.703696 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816628cf-c5b5-4503-ad2e-4e15d9b9e076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:20.721755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-816628cf-c5b5-4503-ad2e-4e15d9b9e076 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:20.722955 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-816628cf-c5b5-4503-ad2e-4e15d9b9e076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:20.733446 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 935/1882] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:20.737507 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92576c38-9c4b-48d3-b904-b04f9e63ee7f None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:20.750309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92576c38-9c4b-48d3-b904-b04f9e63ee7f None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:20.755533 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebff543-da47-4635-93c7-66a9fc6f7119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:20.756306 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebff543-da47-4635-93c7-66a9fc6f7119 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:20.756591 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebff543-da47-4635-93c7-66a9fc6f7119 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:20.757584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebff543-da47-4635-93c7-66a9fc6f7119 admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:20.758534 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebff543-da47-4635-93c7-66a9fc6f7119 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:46:20.758816 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebff543-da47-4635-93c7-66a9fc6f7119 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:20.760408 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebff543-da47-4635-93c7-66a9fc6f7119 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:20.772729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 948/1883] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:46:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:46:20.778336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816628cf-c5b5-4503-ad2e-4e15d9b9e076 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:20.788611 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816628cf-c5b5-4503-ad2e-4e15d9b9e076 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:20.793384 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85127b-01a8-415f-ab5e-2a75ecedc1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:20.794061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85127b-01a8-415f-ab5e-2a75ecedc1bf admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:20.794370 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85127b-01a8-415f-ab5e-2a75ecedc1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:20.794651 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85127b-01a8-415f-ab5e-2a75ecedc1bf admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:20.795373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d85127b-01a8-415f-ab5e-2a75ecedc1bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:20.797320 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d85127b-01a8-415f-ab5e-2a75ecedc1bf admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:20.823687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 936/1884] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:46:20 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:46:20.827764 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebff543-da47-4635-93c7-66a9fc6f7119 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:20.837783 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebff543-da47-4635-93c7-66a9fc6f7119 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:20.892302 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdac63e-1d4a-420c-9601-5132bd62a92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:20.893065 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdac63e-1d4a-420c-9601-5132bd62a92f admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:20.893394 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdac63e-1d4a-420c-9601-5132bd62a92f admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:20.893726 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdac63e-1d4a-420c-9601-5132bd62a92f admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:20.894423 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdac63e-1d4a-420c-9601-5132bd62a92f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:20.896312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdac63e-1d4a-420c-9601-5132bd62a92f admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:20.920059 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 949/1885] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:46:20 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:46:20.923962 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85127b-01a8-415f-ab5e-2a75ecedc1bf admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:20.934965 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85127b-01a8-415f-ab5e-2a75ecedc1bf admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:20.939515 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63897067-70a3-452e-97b5-a1fffc3923de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:20.940170 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63897067-70a3-452e-97b5-a1fffc3923de admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:20.940452 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63897067-70a3-452e-97b5-a1fffc3923de admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:20.940728 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63897067-70a3-452e-97b5-a1fffc3923de admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:20.941479 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63897067-70a3-452e-97b5-a1fffc3923de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:20.943099 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63897067-70a3-452e-97b5-a1fffc3923de admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:20.953947 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 937/1886] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:46:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:46:20.957782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdac63e-1d4a-420c-9601-5132bd62a92f admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:20.967698 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdac63e-1d4a-420c-9601-5132bd62a92f admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:20.972324 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a4ed6-799a-44b0-8727-b31063d4df29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:20.972976 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a4ed6-799a-44b0-8727-b31063d4df29 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:20.973293 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a4ed6-799a-44b0-8727-b31063d4df29 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:20.973567 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a4ed6-799a-44b0-8727-b31063d4df29 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:20.974320 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5a4ed6-799a-44b0-8727-b31063d4df29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:20.976761 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5a4ed6-799a-44b0-8727-b31063d4df29 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:20.990526 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 950/1887] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:46:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:46:20.993063 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63897067-70a3-452e-97b5-a1fffc3923de admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:21.003329 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63897067-70a3-452e-97b5-a1fffc3923de admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:21.008476 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783654a0-26fe-472e-98ef-ae302fc7159e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:21.009428 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783654a0-26fe-472e-98ef-ae302fc7159e admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:21.009717 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783654a0-26fe-472e-98ef-ae302fc7159e admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:21.010026 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783654a0-26fe-472e-98ef-ae302fc7159e admin admin] PATH_INFO: `/v3/projects/1727f66e37f94cc6971b574c8635e078/users/53d00e4198e240c8a5c2fbe8bd241b45/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:21.027709 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783654a0-26fe-472e-98ef-ae302fc7159e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1727f66e37f94cc6971b574c8635e078, user_id=53d00e4198e240c8a5c2fbe8bd241b45, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:21.030512 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783654a0-26fe-472e-98ef-ae302fc7159e admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:21.048366 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 938/1888] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:46:20 2022] PUT /identity/v3/projects/1727f66e37f94cc6971b574c8635e078/users/53d00e4198e240c8a5c2fbe8bd241b45/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:46:21.050183 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34bd37d-4e0d-4ee1-b233-79527e4920a8 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:21.050512 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34bd37d-4e0d-4ee1-b233-79527e4920a8 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:21.050804 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34bd37d-4e0d-4ee1-b233-79527e4920a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:21.068988 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-f34bd37d-4e0d-4ee1-b233-79527e4920a8 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:21.070403 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-f34bd37d-4e0d-4ee1-b233-79527e4920a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:21.137078 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 951/1889] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:46:21.143956 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783654a0-26fe-472e-98ef-ae302fc7159e admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:21.146338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-783654a0-26fe-472e-98ef-ae302fc7159e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:21.214980 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783654a0-26fe-472e-98ef-ae302fc7159e admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:21.218675 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0019f4-cac3-46f1-98c6-e32ed77f6ebd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:21.219370 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0019f4-cac3-46f1-98c6-e32ed77f6ebd service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:21.219679 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0019f4-cac3-46f1-98c6-e32ed77f6ebd service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:21.219979 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0019f4-cac3-46f1-98c6-e32ed77f6ebd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:21.229602 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0019f4-cac3-46f1-98c6-e32ed77f6ebd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:21.231054 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0019f4-cac3-46f1-98c6-e32ed77f6ebd service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:21.241647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 939/1890] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:46:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 13:46:26.003096 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec81b04-04f3-41bc-b759-6a4e217b14c9 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:26.003597 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec81b04-04f3-41bc-b759-6a4e217b14c9 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:26.003776 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec81b04-04f3-41bc-b759-6a4e217b14c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:26.011817 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:26.037921 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-9ec81b04-04f3-41bc-b759-6a4e217b14c9 None None] MFA Rules not processed for user `53d00e4198e240c8a5c2fbe8bd241b45`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:26.039532 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-9ec81b04-04f3-41bc-b759-6a4e217b14c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:26.114175 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 952/1891] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:26.119003 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0019f4-cac3-46f1-98c6-e32ed77f6ebd service neutron] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:26.129269 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0019f4-cac3-46f1-98c6-e32ed77f6ebd service neutron] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:26.136293 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1adccb-7c6b-422c-b5f7-d1e36de15148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:26.136963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1adccb-7c6b-422c-b5f7-d1e36de15148 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:26.137395 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1adccb-7c6b-422c-b5f7-d1e36de15148 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:26.137684 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1adccb-7c6b-422c-b5f7-d1e36de15148 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:26.138387 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1adccb-7c6b-422c-b5f7-d1e36de15148 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:26.140310 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1adccb-7c6b-422c-b5f7-d1e36de15148 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:26.173250 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 940/1892] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:46:26 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:46:26.177352 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec81b04-04f3-41bc-b759-6a4e217b14c9 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:26.196679 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec81b04-04f3-41bc-b759-6a4e217b14c9 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:26.279047 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f30d7-ba86-48cd-abc6-8047547f6711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:26.279949 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f30d7-ba86-48cd-abc6-8047547f6711 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:26.280239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f30d7-ba86-48cd-abc6-8047547f6711 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:26.280517 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f30d7-ba86-48cd-abc6-8047547f6711 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:26.281263 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82f30d7-ba86-48cd-abc6-8047547f6711 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:26.283234 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82f30d7-ba86-48cd-abc6-8047547f6711 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:26.315623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 953/1893] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:46:26 2022] POST /identity/v3/users => generated 292 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:46:26.319752 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1adccb-7c6b-422c-b5f7-d1e36de15148 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:26.331763 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1adccb-7c6b-422c-b5f7-d1e36de15148 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:26.338401 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f01a59-00a9-4cbb-983b-60e717245e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:26.339306 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f01a59-00a9-4cbb-983b-60e717245e25 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:26.340484 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f01a59-00a9-4cbb-983b-60e717245e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:26.340778 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f01a59-00a9-4cbb-983b-60e717245e25 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:26.341562 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f01a59-00a9-4cbb-983b-60e717245e25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:26.343248 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f01a59-00a9-4cbb-983b-60e717245e25 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:26.358487 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 941/1894] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:46:26.360969 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f30d7-ba86-48cd-abc6-8047547f6711 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:26.378138 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f30d7-ba86-48cd-abc6-8047547f6711 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:26.384044 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb3dff-5626-4029-b88e-c818bb78f8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:26.385196 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb3dff-5626-4029-b88e-c818bb78f8de admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:26.385631 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb3dff-5626-4029-b88e-c818bb78f8de admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:26.386084 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb3dff-5626-4029-b88e-c818bb78f8de admin admin] PATH_INFO: `/v3/projects/0a185cea242c43d3a0919fc48a32f681/users/1abf4a9a40b94ed88ac15703cd5dc36f/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:26.408034 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeb3dff-5626-4029-b88e-c818bb78f8de admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a185cea242c43d3a0919fc48a32f681, user_id=1abf4a9a40b94ed88ac15703cd5dc36f, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:26.410971 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeb3dff-5626-4029-b88e-c818bb78f8de admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:26.442173 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 954/1895] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:46:26 2022] PUT /identity/v3/projects/0a185cea242c43d3a0919fc48a32f681/users/1abf4a9a40b94ed88ac15703cd5dc36f/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:26.446616 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f01a59-00a9-4cbb-983b-60e717245e25 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:26.456669 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f01a59-00a9-4cbb-983b-60e717245e25 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:26.512755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:26.520605 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ededfec-34b4-430a-a902-aaee8380484f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:26.521352 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ededfec-34b4-430a-a902-aaee8380484f admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:26.521648 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ededfec-34b4-430a-a902-aaee8380484f admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:26.523056 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ededfec-34b4-430a-a902-aaee8380484f admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:26.524145 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ededfec-34b4-430a-a902-aaee8380484f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:26.526446 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ededfec-34b4-430a-a902-aaee8380484f admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:26.540568 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 942/1896] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:46:26.542184 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb3dff-5626-4029-b88e-c818bb78f8de admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:26.555259 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb3dff-5626-4029-b88e-c818bb78f8de admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:26.559793 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d983e0-6b3a-4a12-88aa-60a8973288ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:26.560704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d983e0-6b3a-4a12-88aa-60a8973288ec admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:26.560987 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d983e0-6b3a-4a12-88aa-60a8973288ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:26.561295 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d983e0-6b3a-4a12-88aa-60a8973288ec admin admin] PATH_INFO: `/v3/projects/0a185cea242c43d3a0919fc48a32f681/users/1abf4a9a40b94ed88ac15703cd5dc36f/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:26.563248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d983e0-6b3a-4a12-88aa-60a8973288ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a185cea242c43d3a0919fc48a32f681, user_id=1abf4a9a40b94ed88ac15703cd5dc36f, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:26.565895 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d983e0-6b3a-4a12-88aa-60a8973288ec admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:26.576344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 955/1897] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:46:26 2022] PUT /identity/v3/projects/0a185cea242c43d3a0919fc48a32f681/users/1abf4a9a40b94ed88ac15703cd5dc36f/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:29.517067 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ededfec-34b4-430a-a902-aaee8380484f admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:29.522662 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-4ededfec-34b4-430a-a902-aaee8380484f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:29.555716 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ededfec-34b4-430a-a902-aaee8380484f admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:29.734058 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db24a3bd-ceb5-4370-831c-6899b9037ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:29.734968 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24a3bd-ceb5-4370-831c-6899b9037ade admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:29.735261 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24a3bd-ceb5-4370-831c-6899b9037ade admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:29.735541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24a3bd-ceb5-4370-831c-6899b9037ade admin admin] PATH_INFO: `/v3/users/6dc6496b22704b2991737c5e648c1538` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:29.737971 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db24a3bd-ceb5-4370-831c-6899b9037ade admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dc6496b22704b2991737c5e648c1538)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:29.739928 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db24a3bd-ceb5-4370-831c-6899b9037ade admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:29.842939 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-db24a3bd-ceb5-4370-831c-6899b9037ade admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dc6496b22704b2991737c5e648c1538'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:29.868611 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-db24a3bd-ceb5-4370-831c-6899b9037ade admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dc6496b22704b2991737c5e648c1538'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:29.869790 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-db24a3bd-ceb5-4370-831c-6899b9037ade admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dc6496b22704b2991737c5e648c1538'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:29.891952 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-db24a3bd-ceb5-4370-831c-6899b9037ade admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dc6496b22704b2991737c5e648c1538'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:29.926536 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-db24a3bd-ceb5-4370-831c-6899b9037ade admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dc6496b22704b2991737c5e648c1538'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:29.939860 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 943/1898] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:46:29 2022] DELETE /identity/v3/users/6dc6496b22704b2991737c5e648c1538 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:29.944838 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b963bb-51b4-47b0-8f4f-14737d65e562 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:29.945467 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b963bb-51b4-47b0-8f4f-14737d65e562 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:29.946157 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b963bb-51b4-47b0-8f4f-14737d65e562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:29.988823 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-d0b963bb-51b4-47b0-8f4f-14737d65e562 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:29.997534 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-d0b963bb-51b4-47b0-8f4f-14737d65e562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:30.126690 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:30.218241 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 956/1899] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:30.230175 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db24a3bd-ceb5-4370-831c-6899b9037ade admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:30.232832 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-db24a3bd-ceb5-4370-831c-6899b9037ade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:30.338321 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db24a3bd-ceb5-4370-831c-6899b9037ade admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:30.342867 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a832fb9c-3cf4-426e-ac72-877ab920190d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:30.344279 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832fb9c-3cf4-426e-ac72-877ab920190d service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:30.344703 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832fb9c-3cf4-426e-ac72-877ab920190d service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:30.345229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832fb9c-3cf4-426e-ac72-877ab920190d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:30.361486 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a832fb9c-3cf4-426e-ac72-877ab920190d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:30.363505 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a832fb9c-3cf4-426e-ac72-877ab920190d service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:30.378379 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 944/1900] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:46:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:46:31.092438 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b963bb-51b4-47b0-8f4f-14737d65e562 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:31.094507 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-d0b963bb-51b4-47b0-8f4f-14737d65e562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:31.112808 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b963bb-51b4-47b0-8f4f-14737d65e562 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:31.127434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae94f15-31d6-45fb-92a9-2a777093abe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:31.128441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae94f15-31d6-45fb-92a9-2a777093abe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:31.128769 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae94f15-31d6-45fb-92a9-2a777093abe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:31.129179 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae94f15-31d6-45fb-92a9-2a777093abe0 admin admin] PATH_INFO: `/v3/projects/f280861f92d24260800f66464e691f57` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:31.133970 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae94f15-31d6-45fb-92a9-2a777093abe0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f280861f92d24260800f66464e691f57)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:31.136311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae94f15-31d6-45fb-92a9-2a777093abe0 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:31.351149 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-1ae94f15-31d6-45fb-92a9-2a777093abe0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f280861f92d24260800f66464e691f57', 'request_id': 'req-1ae94f15-31d6-45fb-92a9-2a777093abe0'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:31.371935 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-1ae94f15-31d6-45fb-92a9-2a777093abe0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f280861f92d24260800f66464e691f57', 'request_id': 'req-1ae94f15-31d6-45fb-92a9-2a777093abe0'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:31.396322 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-1ae94f15-31d6-45fb-92a9-2a777093abe0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f280861f92d24260800f66464e691f57', 'request_id': 'req-1ae94f15-31d6-45fb-92a9-2a777093abe0'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:31.423808 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-1ae94f15-31d6-45fb-92a9-2a777093abe0 admin admin] The token cache is being invalidate because project f280861f92d24260800f66464e691f57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:46:31.424109 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-1ae94f15-31d6-45fb-92a9-2a777093abe0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:31.426372 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-1ae94f15-31d6-45fb-92a9-2a777093abe0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:31.433373 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 957/1901] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:46:31 2022] DELETE /identity/v3/projects/f280861f92d24260800f66464e691f57 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:31.434932 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a832fb9c-3cf4-426e-ac72-877ab920190d service neutron] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:31.450033 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a832fb9c-3cf4-426e-ac72-877ab920190d service neutron] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:31.469307 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:31.654198 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6654bf-c8de-42a7-95b0-94640594a388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:31.655357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6654bf-c8de-42a7-95b0-94640594a388 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:31.655933 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6654bf-c8de-42a7-95b0-94640594a388 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:31.656392 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6654bf-c8de-42a7-95b0-94640594a388 admin admin] PATH_INFO: `/v3/users/9293c5a077404693b276048986c7eaea` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:31.660075 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6654bf-c8de-42a7-95b0-94640594a388 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9293c5a077404693b276048986c7eaea)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:31.668361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6654bf-c8de-42a7-95b0-94640594a388 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:31.804923 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-df6654bf-c8de-42a7-95b0-94640594a388 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9293c5a077404693b276048986c7eaea'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:31.821084 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-df6654bf-c8de-42a7-95b0-94640594a388 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9293c5a077404693b276048986c7eaea'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:31.822255 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-df6654bf-c8de-42a7-95b0-94640594a388 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9293c5a077404693b276048986c7eaea'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:31.844395 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-df6654bf-c8de-42a7-95b0-94640594a388 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9293c5a077404693b276048986c7eaea'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:31.885206 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-df6654bf-c8de-42a7-95b0-94640594a388 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9293c5a077404693b276048986c7eaea'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:31.914879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 945/1902] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:46:31 2022] DELETE /identity/v3/users/9293c5a077404693b276048986c7eaea => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:32.690514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa79676-486d-454d-ab4c-3b513e50d7e7 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:32.691043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa79676-486d-454d-ab4c-3b513e50d7e7 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:32.691043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa79676-486d-454d-ab4c-3b513e50d7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:32.744950 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-cfa79676-486d-454d-ab4c-3b513e50d7e7 None None] MFA Rules not processed for user `1abf4a9a40b94ed88ac15703cd5dc36f`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:32.758192 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6654bf-c8de-42a7-95b0-94640594a388 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:32.761154 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-cfa79676-486d-454d-ab4c-3b513e50d7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:32.763152 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-df6654bf-c8de-42a7-95b0-94640594a388 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:32.872051 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6654bf-c8de-42a7-95b0-94640594a388 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:32.888008 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d09fe-7503-40e7-ab47-eb7189db5a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:32.889201 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d09fe-7503-40e7-ab47-eb7189db5a0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:32.890003 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d09fe-7503-40e7-ab47-eb7189db5a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:32.891301 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d09fe-7503-40e7-ab47-eb7189db5a0d admin admin] PATH_INFO: `/v3/projects/5154e8024ab74ee7bb9740ee6d5417d1` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:32.893696 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d09fe-7503-40e7-ab47-eb7189db5a0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5154e8024ab74ee7bb9740ee6d5417d1)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:32.896314 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d09fe-7503-40e7-ab47-eb7189db5a0d admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:32.924287 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 958/1903] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:32 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:32.942955 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa79676-486d-454d-ab4c-3b513e50d7e7 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:32.945329 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-cfa79676-486d-454d-ab4c-3b513e50d7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:32.960023 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:32.984736 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-470d09fe-7503-40e7-ab47-eb7189db5a0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5154e8024ab74ee7bb9740ee6d5417d1', 'request_id': 'req-470d09fe-7503-40e7-ab47-eb7189db5a0d'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:32.998635 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-470d09fe-7503-40e7-ab47-eb7189db5a0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5154e8024ab74ee7bb9740ee6d5417d1', 'request_id': 'req-470d09fe-7503-40e7-ab47-eb7189db5a0d'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:33.015074 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-470d09fe-7503-40e7-ab47-eb7189db5a0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5154e8024ab74ee7bb9740ee6d5417d1', 'request_id': 'req-470d09fe-7503-40e7-ab47-eb7189db5a0d'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:33.043492 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-470d09fe-7503-40e7-ab47-eb7189db5a0d admin admin] The token cache is being invalidate because project 5154e8024ab74ee7bb9740ee6d5417d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:46:33.043859 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-470d09fe-7503-40e7-ab47-eb7189db5a0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:33.044699 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-470d09fe-7503-40e7-ab47-eb7189db5a0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:33.048907 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 946/1904] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:46:32 2022] DELETE /identity/v3/projects/5154e8024ab74ee7bb9740ee6d5417d1 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:46:33.060959 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf00b2-b966-4685-b138-692dcee846af None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:33.061301 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf00b2-b966-4685-b138-692dcee846af None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:33.062668 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf00b2-b966-4685-b138-692dcee846af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:33.064741 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:33.105378 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa79676-486d-454d-ab4c-3b513e50d7e7 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:33.106050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-fabf00b2-b966-4685-b138-692dcee846af None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:33.108344 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-fabf00b2-b966-4685-b138-692dcee846af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:33.218018 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 947/1905] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:33.227065 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf938d-a251-435f-b9cb-ffcc8d0bc242 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:33.228507 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf938d-a251-435f-b9cb-ffcc8d0bc242 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:33.229156 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf938d-a251-435f-b9cb-ffcc8d0bc242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:33.256087 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-40bf938d-a251-435f-b9cb-ffcc8d0bc242 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:33.257298 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64872af6-af75-49ea-ae8f-9b8765ebcc3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:33.258096 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64872af6-af75-49ea-ae8f-9b8765ebcc3e service nova] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:33.258424 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64872af6-af75-49ea-ae8f-9b8765ebcc3e service nova] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:33.258779 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64872af6-af75-49ea-ae8f-9b8765ebcc3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:33.259231 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-40bf938d-a251-435f-b9cb-ffcc8d0bc242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:33.260894 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-64872af6-af75-49ea-ae8f-9b8765ebcc3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:33.278169 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 948/1906] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:46:33.282268 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf938d-a251-435f-b9cb-ffcc8d0bc242 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:33.292538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf938d-a251-435f-b9cb-ffcc8d0bc242 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:33.298581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ace57f-7703-4a01-ad05-22892002cfd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:33.299589 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ace57f-7703-4a01-ad05-22892002cfd9 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:33.300082 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ace57f-7703-4a01-ad05-22892002cfd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:33.300573 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ace57f-7703-4a01-ad05-22892002cfd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:33.302205 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ace57f-7703-4a01-ad05-22892002cfd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:46:33.302741 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ace57f-7703-4a01-ad05-22892002cfd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:33.304500 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ace57f-7703-4a01-ad05-22892002cfd9 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:33.323799 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 949/1907] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:46:33.331470 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ace57f-7703-4a01-ad05-22892002cfd9 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:33.342597 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ace57f-7703-4a01-ad05-22892002cfd9 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:33.349350 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91753c8c-1850-4942-940a-229aab649ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:33.350674 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91753c8c-1850-4942-940a-229aab649ee1 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:33.351114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91753c8c-1850-4942-940a-229aab649ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:33.351494 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91753c8c-1850-4942-940a-229aab649ee1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:33.352312 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91753c8c-1850-4942-940a-229aab649ee1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:33.354497 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91753c8c-1850-4942-940a-229aab649ee1 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:33.409047 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64872af6-af75-49ea-ae8f-9b8765ebcc3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:33.414589 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64872af6-af75-49ea-ae8f-9b8765ebcc3e service nova] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:33.428136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 950/1908] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:46:33 2022] POST /identity/v3/projects => generated 371 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:46:33.429291 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91753c8c-1850-4942-940a-229aab649ee1 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:33.446140 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 959/1909] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:46:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 13:46:33.552248 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91753c8c-1850-4942-940a-229aab649ee1 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:33.566854 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ee977-6e04-4b59-8a6a-35d7d60a16d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:33.567549 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ee977-6e04-4b59-8a6a-35d7d60a16d4 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:33.567840 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ee977-6e04-4b59-8a6a-35d7d60a16d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:33.568118 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ee977-6e04-4b59-8a6a-35d7d60a16d4 admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:33.571649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ee977-6e04-4b59-8a6a-35d7d60a16d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:33.576914 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ee977-6e04-4b59-8a6a-35d7d60a16d4 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:33.634980 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 951/1910] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:46:33 2022] POST /identity/v3/users => generated 286 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:46:33.639192 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64872af6-af75-49ea-ae8f-9b8765ebcc3e service nova] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:33.656094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64872af6-af75-49ea-ae8f-9b8765ebcc3e service nova] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:33.664134 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac272ab3-0610-4336-b9a0-fbc975da584b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:33.665030 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac272ab3-0610-4336-b9a0-fbc975da584b admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:33.665804 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac272ab3-0610-4336-b9a0-fbc975da584b admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:33.666335 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac272ab3-0610-4336-b9a0-fbc975da584b admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:33.667185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac272ab3-0610-4336-b9a0-fbc975da584b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:33.670049 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac272ab3-0610-4336-b9a0-fbc975da584b admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:33.687282 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 960/1911] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:46:33.691188 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ee977-6e04-4b59-8a6a-35d7d60a16d4 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:33.705415 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ee977-6e04-4b59-8a6a-35d7d60a16d4 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:33.713114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dade90-b982-4cb6-9712-485cffb76fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:33.714069 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dade90-b982-4cb6-9712-485cffb76fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:33.714508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dade90-b982-4cb6-9712-485cffb76fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:33.715360 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dade90-b982-4cb6-9712-485cffb76fa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:33.717664 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dade90-b982-4cb6-9712-485cffb76fa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:33.720324 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dade90-b982-4cb6-9712-485cffb76fa0 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:33.736286 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 952/1912] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:46:33.737781 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac272ab3-0610-4336-b9a0-fbc975da584b admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:33.757818 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac272ab3-0610-4336-b9a0-fbc975da584b admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:33.764042 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1597213-d7ab-4650-90e4-f9112dff1517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:33.765475 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1597213-d7ab-4650-90e4-f9112dff1517 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:33.766224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1597213-d7ab-4650-90e4-f9112dff1517 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:33.766826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1597213-d7ab-4650-90e4-f9112dff1517 admin admin] PATH_INFO: `/v3/projects/2f9f86fa910c4054b31927ba26ea96b9/users/050d5eb22b2c45ee85eb8c118b9ef21e/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:33.799368 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1597213-d7ab-4650-90e4-f9112dff1517 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f9f86fa910c4054b31927ba26ea96b9, user_id=050d5eb22b2c45ee85eb8c118b9ef21e, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:33.803314 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1597213-d7ab-4650-90e4-f9112dff1517 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:33.836979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 961/1913] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:46:33 2022] PUT /identity/v3/projects/2f9f86fa910c4054b31927ba26ea96b9/users/050d5eb22b2c45ee85eb8c118b9ef21e/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:46:33.840058 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51c0e9-dbbd-43ec-aa44-c8afdf9e7abc None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:33.840820 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51c0e9-dbbd-43ec-aa44-c8afdf9e7abc None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:33.841464 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51c0e9-dbbd-43ec-aa44-c8afdf9e7abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:33.848428 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:33.869293 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-0c51c0e9-dbbd-43ec-aa44-c8afdf9e7abc None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:33.870546 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-0c51c0e9-dbbd-43ec-aa44-c8afdf9e7abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:33.947770 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 953/1914] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:33.955676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1597213-d7ab-4650-90e4-f9112dff1517 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:33.960185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-a1597213-d7ab-4650-90e4-f9112dff1517 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:34.054037 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1597213-d7ab-4650-90e4-f9112dff1517 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:34.060848 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e76053d-cc55-47b9-9881-d3a1316ddda0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:34.064981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76053d-cc55-47b9-9881-d3a1316ddda0 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:34.065412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76053d-cc55-47b9-9881-d3a1316ddda0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:34.065796 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76053d-cc55-47b9-9881-d3a1316ddda0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:34.081273 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e76053d-cc55-47b9-9881-d3a1316ddda0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:34.082873 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e76053d-cc55-47b9-9881-d3a1316ddda0 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:34.094419 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:34.096956 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 962/1915] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:46:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:46:35.103521 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1658ab-8c63-4721-82d5-6b6c7fc0dcfd None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:35.104087 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1658ab-8c63-4721-82d5-6b6c7fc0dcfd None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:35.104087 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1658ab-8c63-4721-82d5-6b6c7fc0dcfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:35.130309 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-8f1658ab-8c63-4721-82d5-6b6c7fc0dcfd None None] MFA Rules not processed for user `53d00e4198e240c8a5c2fbe8bd241b45`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:35.132674 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-8f1658ab-8c63-4721-82d5-6b6c7fc0dcfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:35.223522 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 954/1916] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:35.392202 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69814418-6c80-498f-8934-899219913d0d None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:35.392773 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69814418-6c80-498f-8934-899219913d0d None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:35.392773 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69814418-6c80-498f-8934-899219913d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:35.430118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-69814418-6c80-498f-8934-899219913d0d None None] MFA Rules not processed for user `050d5eb22b2c45ee85eb8c118b9ef21e`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:35.432856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-69814418-6c80-498f-8934-899219913d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:35.554664 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 963/1917] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:35.673561 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1658ab-8c63-4721-82d5-6b6c7fc0dcfd None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:35.679296 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-8f1658ab-8c63-4721-82d5-6b6c7fc0dcfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:35.831503 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1658ab-8c63-4721-82d5-6b6c7fc0dcfd None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:35.837934 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044fc26b-b0f5-4683-8f58-e30474e0e805 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:35.838609 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fc26b-b0f5-4683-8f58-e30474e0e805 service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:35.838898 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fc26b-b0f5-4683-8f58-e30474e0e805 service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:35.839174 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fc26b-b0f5-4683-8f58-e30474e0e805 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:35.851744 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044fc26b-b0f5-4683-8f58-e30474e0e805 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:35.853478 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044fc26b-b0f5-4683-8f58-e30474e0e805 service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:35.868251 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 955/1918] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:46:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:46:37.053890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69814418-6c80-498f-8934-899219913d0d None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:37.056765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-69814418-6c80-498f-8934-899219913d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:37.081814 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69814418-6c80-498f-8934-899219913d0d None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:37.088552 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe56276-95c1-4b3a-8be5-70f5342f78b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:37.090116 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe56276-95c1-4b3a-8be5-70f5342f78b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:37.099016 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe56276-95c1-4b3a-8be5-70f5342f78b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:37.099948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe56276-95c1-4b3a-8be5-70f5342f78b1 admin admin] PATH_INFO: `/v3/users/9d13fb415ee54beea237ed2258c89691` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:37.103027 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe56276-95c1-4b3a-8be5-70f5342f78b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d13fb415ee54beea237ed2258c89691)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:37.108137 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe56276-95c1-4b3a-8be5-70f5342f78b1 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:37.188030 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-ffe56276-95c1-4b3a-8be5-70f5342f78b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d13fb415ee54beea237ed2258c89691'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:37.204883 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-ffe56276-95c1-4b3a-8be5-70f5342f78b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d13fb415ee54beea237ed2258c89691'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:37.214775 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-ffe56276-95c1-4b3a-8be5-70f5342f78b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d13fb415ee54beea237ed2258c89691'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:37.215828 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-ffe56276-95c1-4b3a-8be5-70f5342f78b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d13fb415ee54beea237ed2258c89691'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:37.229957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-ffe56276-95c1-4b3a-8be5-70f5342f78b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d13fb415ee54beea237ed2258c89691'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:37.238015 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 964/1919] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:46:37 2022] DELETE /identity/v3/users/9d13fb415ee54beea237ed2258c89691 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:37.243103 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caeb8cb-1790-413c-99f7-431cb1cf079e None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:37.243657 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caeb8cb-1790-413c-99f7-431cb1cf079e None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:37.243847 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caeb8cb-1790-413c-99f7-431cb1cf079e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:37.264914 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-6caeb8cb-1790-413c-99f7-431cb1cf079e None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:37.266394 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-6caeb8cb-1790-413c-99f7-431cb1cf079e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:37.322044 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:37.343652 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 956/1920] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:37.462270 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe56276-95c1-4b3a-8be5-70f5342f78b1 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:37.463981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-ffe56276-95c1-4b3a-8be5-70f5342f78b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:37.626987 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe56276-95c1-4b3a-8be5-70f5342f78b1 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:37.631633 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58514505-a004-4e16-a0b4-5337d660f164 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:37.632935 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58514505-a004-4e16-a0b4-5337d660f164 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:37.633301 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58514505-a004-4e16-a0b4-5337d660f164 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:37.633581 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58514505-a004-4e16-a0b4-5337d660f164 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:37.661078 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58514505-a004-4e16-a0b4-5337d660f164 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:37.679134 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58514505-a004-4e16-a0b4-5337d660f164 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:37.680003 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a18215-1f8e-4795-8f98-19f66fa5bc07 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:37.680624 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a18215-1f8e-4795-8f98-19f66fa5bc07 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:37.680980 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a18215-1f8e-4795-8f98-19f66fa5bc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:37.691818 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 965/1921] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:46:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:46:37.729462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-c3a18215-1f8e-4795-8f98-19f66fa5bc07 None None] MFA Rules not processed for user `050d5eb22b2c45ee85eb8c118b9ef21e`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:37.731622 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-c3a18215-1f8e-4795-8f98-19f66fa5bc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:37.985590 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 957/1922] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:38.766914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58514505-a004-4e16-a0b4-5337d660f164 service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:38.768950 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-58514505-a004-4e16-a0b4-5337d660f164 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:38.778054 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:38.783681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58514505-a004-4e16-a0b4-5337d660f164 service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:38.790767 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b309a6-6654-4f80-99de-38328a07ca64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:38.791719 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b309a6-6654-4f80-99de-38328a07ca64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:38.792606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b309a6-6654-4f80-99de-38328a07ca64 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:38.793157 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b309a6-6654-4f80-99de-38328a07ca64 admin admin] PATH_INFO: `/v3/projects/ba428e3a1e094f7c83fae3f3d5f6e8c2` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:38.795778 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b309a6-6654-4f80-99de-38328a07ca64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba428e3a1e094f7c83fae3f3d5f6e8c2)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:38.800020 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b309a6-6654-4f80-99de-38328a07ca64 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:38.886190 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-09b309a6-6654-4f80-99de-38328a07ca64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba428e3a1e094f7c83fae3f3d5f6e8c2', 'request_id': 'req-09b309a6-6654-4f80-99de-38328a07ca64'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:38.893025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-09b309a6-6654-4f80-99de-38328a07ca64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba428e3a1e094f7c83fae3f3d5f6e8c2', 'request_id': 'req-09b309a6-6654-4f80-99de-38328a07ca64'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:38.905151 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-09b309a6-6654-4f80-99de-38328a07ca64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba428e3a1e094f7c83fae3f3d5f6e8c2', 'request_id': 'req-09b309a6-6654-4f80-99de-38328a07ca64'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:38.920058 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-09b309a6-6654-4f80-99de-38328a07ca64 admin admin] The token cache is being invalidate because project ba428e3a1e094f7c83fae3f3d5f6e8c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:46:38.920405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-09b309a6-6654-4f80-99de-38328a07ca64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:38.921122 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-09b309a6-6654-4f80-99de-38328a07ca64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:38.923069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 966/1923] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:46:38 2022] DELETE /identity/v3/projects/ba428e3a1e094f7c83fae3f3d5f6e8c2 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:38.927289 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a18215-1f8e-4795-8f98-19f66fa5bc07 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:38.951626 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a18215-1f8e-4795-8f98-19f66fa5bc07 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:39.046500 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb93c0-657a-4443-8f4d-668b5875b2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:39.047476 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb93c0-657a-4443-8f4d-668b5875b2d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:39.047832 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb93c0-657a-4443-8f4d-668b5875b2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:39.048177 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb93c0-657a-4443-8f4d-668b5875b2d8 admin admin] PATH_INFO: `/v3/users/dd7a4a682d344a6e81cedf1c1d1a19e1` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:39.049849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcb93c0-657a-4443-8f4d-668b5875b2d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd7a4a682d344a6e81cedf1c1d1a19e1)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:39.054662 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcb93c0-657a-4443-8f4d-668b5875b2d8 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:39.173749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-cdcb93c0-657a-4443-8f4d-668b5875b2d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd7a4a682d344a6e81cedf1c1d1a19e1'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:39.188288 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-cdcb93c0-657a-4443-8f4d-668b5875b2d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd7a4a682d344a6e81cedf1c1d1a19e1'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:39.189683 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-cdcb93c0-657a-4443-8f4d-668b5875b2d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd7a4a682d344a6e81cedf1c1d1a19e1'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:39.212887 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-cdcb93c0-657a-4443-8f4d-668b5875b2d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd7a4a682d344a6e81cedf1c1d1a19e1'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:39.237967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-cdcb93c0-657a-4443-8f4d-668b5875b2d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd7a4a682d344a6e81cedf1c1d1a19e1'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:39.246831 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 958/1924] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:46:38 2022] DELETE /identity/v3/users/dd7a4a682d344a6e81cedf1c1d1a19e1 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:39.673315 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b309a6-6654-4f80-99de-38328a07ca64 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:39.675186 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-09b309a6-6654-4f80-99de-38328a07ca64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:39.699975 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b309a6-6654-4f80-99de-38328a07ca64 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:39.785070 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ccbd6-9c75-4a79-a7c1-7fa1ff95ed76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:39.786011 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ccbd6-9c75-4a79-a7c1-7fa1ff95ed76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:39.786370 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ccbd6-9c75-4a79-a7c1-7fa1ff95ed76 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:39.786905 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ccbd6-9c75-4a79-a7c1-7fa1ff95ed76 admin admin] PATH_INFO: `/v3/projects/7f26915cd1f540a6a3ecc3ff1e370f82` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:39.788931 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995ccbd6-9c75-4a79-a7c1-7fa1ff95ed76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f26915cd1f540a6a3ecc3ff1e370f82)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:39.791089 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995ccbd6-9c75-4a79-a7c1-7fa1ff95ed76 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:39.856436 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-995ccbd6-9c75-4a79-a7c1-7fa1ff95ed76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f26915cd1f540a6a3ecc3ff1e370f82', 'request_id': 'req-995ccbd6-9c75-4a79-a7c1-7fa1ff95ed76'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:39.862101 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-995ccbd6-9c75-4a79-a7c1-7fa1ff95ed76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f26915cd1f540a6a3ecc3ff1e370f82', 'request_id': 'req-995ccbd6-9c75-4a79-a7c1-7fa1ff95ed76'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:39.873596 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-995ccbd6-9c75-4a79-a7c1-7fa1ff95ed76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f26915cd1f540a6a3ecc3ff1e370f82', 'request_id': 'req-995ccbd6-9c75-4a79-a7c1-7fa1ff95ed76'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:39.892600 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-995ccbd6-9c75-4a79-a7c1-7fa1ff95ed76 admin admin] The token cache is being invalidate because project 7f26915cd1f540a6a3ecc3ff1e370f82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:46:39.892600 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-995ccbd6-9c75-4a79-a7c1-7fa1ff95ed76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:39.892600 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-995ccbd6-9c75-4a79-a7c1-7fa1ff95ed76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:39.897408 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 967/1925] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:46:39 2022] DELETE /identity/v3/projects/7f26915cd1f540a6a3ecc3ff1e370f82 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:39.899474 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4cbd2-4840-4f9d-82f3-565d27fc8624 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:39.899761 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4cbd2-4840-4f9d-82f3-565d27fc8624 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:39.900035 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4cbd2-4840-4f9d-82f3-565d27fc8624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:39.925651 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-e9a4cbd2-4840-4f9d-82f3-565d27fc8624 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:39.927252 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-e9a4cbd2-4840-4f9d-82f3-565d27fc8624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:39.937172 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:39.995596 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 959/1926] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:40.002800 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9685d-be8d-4cdf-8c3e-6f29a0dfe73e None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:40.003162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9685d-be8d-4cdf-8c3e-6f29a0dfe73e None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:40.003505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9685d-be8d-4cdf-8c3e-6f29a0dfe73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:40.022644 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-6bc9685d-be8d-4cdf-8c3e-6f29a0dfe73e None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:40.024065 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-6bc9685d-be8d-4cdf-8c3e-6f29a0dfe73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:40.032617 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 968/1927] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:40.036846 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4cbd2-4840-4f9d-82f3-565d27fc8624 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:40.046935 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4cbd2-4840-4f9d-82f3-565d27fc8624 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:40.051740 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de756891-b18e-49d0-a5ff-527203117da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:40.052394 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de756891-b18e-49d0-a5ff-527203117da8 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:40.052701 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de756891-b18e-49d0-a5ff-527203117da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:40.054622 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de756891-b18e-49d0-a5ff-527203117da8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:40.054622 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de756891-b18e-49d0-a5ff-527203117da8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:46:40.054622 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de756891-b18e-49d0-a5ff-527203117da8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:40.055618 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de756891-b18e-49d0-a5ff-527203117da8 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:40.067597 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 960/1928] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:46:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:46:40.071883 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc9685d-be8d-4cdf-8c3e-6f29a0dfe73e None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:40.081024 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc9685d-be8d-4cdf-8c3e-6f29a0dfe73e None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:40.086406 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278f6f5-ca44-49a5-b758-8a0910051740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:40.088738 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278f6f5-ca44-49a5-b758-8a0910051740 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:40.088738 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278f6f5-ca44-49a5-b758-8a0910051740 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:40.088738 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278f6f5-ca44-49a5-b758-8a0910051740 admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:40.088738 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278f6f5-ca44-49a5-b758-8a0910051740 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:40.090174 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278f6f5-ca44-49a5-b758-8a0910051740 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:40.094488 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:40.123648 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 969/1929] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:46:40 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:46:40.127835 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de756891-b18e-49d0-a5ff-527203117da8 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:40.139098 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de756891-b18e-49d0-a5ff-527203117da8 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:40.197569 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c174a01-88f3-4872-8acf-0b80975094b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:40.198276 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c174a01-88f3-4872-8acf-0b80975094b7 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:40.198562 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c174a01-88f3-4872-8acf-0b80975094b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:40.198847 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c174a01-88f3-4872-8acf-0b80975094b7 admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:40.199593 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c174a01-88f3-4872-8acf-0b80975094b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:40.201610 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c174a01-88f3-4872-8acf-0b80975094b7 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:40.227303 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 961/1930] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:46:40 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:46:40.228923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278f6f5-ca44-49a5-b758-8a0910051740 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:40.240626 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278f6f5-ca44-49a5-b758-8a0910051740 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:40.245779 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd965c3-4117-4e10-8c8b-b6928dbfe6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:40.248593 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd965c3-4117-4e10-8c8b-b6928dbfe6d3 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:40.248593 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd965c3-4117-4e10-8c8b-b6928dbfe6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:40.248593 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd965c3-4117-4e10-8c8b-b6928dbfe6d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:40.248593 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd965c3-4117-4e10-8c8b-b6928dbfe6d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:40.249303 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd965c3-4117-4e10-8c8b-b6928dbfe6d3 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:40.260148 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 970/1931] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:46:40.264361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c174a01-88f3-4872-8acf-0b80975094b7 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:40.275536 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c174a01-88f3-4872-8acf-0b80975094b7 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:40.280297 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b359fddc-b50c-4de5-b6d0-e6cec1ee70f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:40.281071 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359fddc-b50c-4de5-b6d0-e6cec1ee70f7 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:40.281425 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359fddc-b50c-4de5-b6d0-e6cec1ee70f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:40.281770 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359fddc-b50c-4de5-b6d0-e6cec1ee70f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:40.282585 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b359fddc-b50c-4de5-b6d0-e6cec1ee70f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:40.284254 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b359fddc-b50c-4de5-b6d0-e6cec1ee70f7 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:40.305306 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 962/1932] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:46:40.306526 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd965c3-4117-4e10-8c8b-b6928dbfe6d3 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:40.316526 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd965c3-4117-4e10-8c8b-b6928dbfe6d3 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:40.322416 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f0dcf-5217-40fc-9a0c-0211297a0fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:40.323328 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f0dcf-5217-40fc-9a0c-0211297a0fe6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:40.323613 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f0dcf-5217-40fc-9a0c-0211297a0fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:40.323887 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f0dcf-5217-40fc-9a0c-0211297a0fe6 admin admin] PATH_INFO: `/v3/projects/417719f21b7549a2841ab0fa703fa1da/users/3ba863143140442abba9ec4d249ee432/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:40.338733 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7f0dcf-5217-40fc-9a0c-0211297a0fe6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=417719f21b7549a2841ab0fa703fa1da, user_id=3ba863143140442abba9ec4d249ee432, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:40.341387 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7f0dcf-5217-40fc-9a0c-0211297a0fe6 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:40.354819 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 971/1933] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:46:40 2022] PUT /identity/v3/projects/417719f21b7549a2841ab0fa703fa1da/users/3ba863143140442abba9ec4d249ee432/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:40.365839 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee4352-a81d-4fff-a3bd-84020a78f656 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:40.366129 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee4352-a81d-4fff-a3bd-84020a78f656 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:40.366486 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee4352-a81d-4fff-a3bd-84020a78f656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:40.392869 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-f7ee4352-a81d-4fff-a3bd-84020a78f656 None None] MFA Rules not processed for user `3ba863143140442abba9ec4d249ee432`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:40.394094 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-f7ee4352-a81d-4fff-a3bd-84020a78f656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:40.464590 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 963/1934] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:40 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:40.469224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f0dcf-5217-40fc-9a0c-0211297a0fe6 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:40.535864 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f0dcf-5217-40fc-9a0c-0211297a0fe6 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:40.541745 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda4a88-0268-4b26-ade2-ccf23219ae65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:40.542418 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda4a88-0268-4b26-ade2-ccf23219ae65 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:40.542698 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda4a88-0268-4b26-ade2-ccf23219ae65 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:40.542979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda4a88-0268-4b26-ade2-ccf23219ae65 admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:40.543691 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afda4a88-0268-4b26-ade2-ccf23219ae65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:40.545612 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afda4a88-0268-4b26-ade2-ccf23219ae65 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:40.576190 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 972/1935] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:46:40 2022] POST /identity/v3/projects => generated 363 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:46:40.585172 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee4352-a81d-4fff-a3bd-84020a78f656 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:40.597294 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee4352-a81d-4fff-a3bd-84020a78f656 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:40.605681 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:40.667328 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b08cbe0-ee3d-4fde-8689-990a0fd2fb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:40.667907 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b08cbe0-ee3d-4fde-8689-990a0fd2fb63 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:40.668205 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b08cbe0-ee3d-4fde-8689-990a0fd2fb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:40.668451 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b08cbe0-ee3d-4fde-8689-990a0fd2fb63 admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:40.669179 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b08cbe0-ee3d-4fde-8689-990a0fd2fb63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:40.671170 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b08cbe0-ee3d-4fde-8689-990a0fd2fb63 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:40.694022 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 964/1936] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:46:40 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:46:40.698310 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda4a88-0268-4b26-ade2-ccf23219ae65 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:40.710209 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda4a88-0268-4b26-ade2-ccf23219ae65 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:40.714973 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c602d80-8ee7-4ce7-b4f7-2f98497e514c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:40.715687 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c602d80-8ee7-4ce7-b4f7-2f98497e514c admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:40.715967 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c602d80-8ee7-4ce7-b4f7-2f98497e514c admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:40.716243 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c602d80-8ee7-4ce7-b4f7-2f98497e514c admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:40.716974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c602d80-8ee7-4ce7-b4f7-2f98497e514c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:40.718717 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c602d80-8ee7-4ce7-b4f7-2f98497e514c admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:40.734160 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 973/1937] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:46:40.738112 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b08cbe0-ee3d-4fde-8689-990a0fd2fb63 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:40.751730 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b08cbe0-ee3d-4fde-8689-990a0fd2fb63 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:40.760291 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a4a91-9314-493c-b0e0-4b15d2864c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:40.761302 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a4a91-9314-493c-b0e0-4b15d2864c2b admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:40.761652 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a4a91-9314-493c-b0e0-4b15d2864c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:40.762013 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a4a91-9314-493c-b0e0-4b15d2864c2b admin admin] PATH_INFO: `/v3/projects/3d24aef0112946b1909ffa96e2ef154e/users/8cad68cf07ad434f8c8d415f1c9f8879/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:40.780599 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a4a91-9314-493c-b0e0-4b15d2864c2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d24aef0112946b1909ffa96e2ef154e, user_id=8cad68cf07ad434f8c8d415f1c9f8879, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:40.783359 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a4a91-9314-493c-b0e0-4b15d2864c2b admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:40.800879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 965/1938] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:46:40 2022] PUT /identity/v3/projects/3d24aef0112946b1909ffa96e2ef154e/users/8cad68cf07ad434f8c8d415f1c9f8879/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:46:40.805261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c602d80-8ee7-4ce7-b4f7-2f98497e514c admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:40.815711 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c602d80-8ee7-4ce7-b4f7-2f98497e514c admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:40.875528 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0785f4f-6ba3-4107-a3e3-626eb99e8865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:40.876201 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0785f4f-6ba3-4107-a3e3-626eb99e8865 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:40.876487 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0785f4f-6ba3-4107-a3e3-626eb99e8865 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:40.876767 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0785f4f-6ba3-4107-a3e3-626eb99e8865 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:40.877531 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0785f4f-6ba3-4107-a3e3-626eb99e8865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:40.879120 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0785f4f-6ba3-4107-a3e3-626eb99e8865 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:40.889610 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 974/1939] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:46:40.893818 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a4a91-9314-493c-b0e0-4b15d2864c2b admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:40.905777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a4a91-9314-493c-b0e0-4b15d2864c2b admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:40.911479 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4248924-2b64-416c-a29d-a5ba90c926a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:40.912979 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4248924-2b64-416c-a29d-a5ba90c926a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:40.913291 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4248924-2b64-416c-a29d-a5ba90c926a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:40.913571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4248924-2b64-416c-a29d-a5ba90c926a5 admin admin] PATH_INFO: `/v3/projects/3d24aef0112946b1909ffa96e2ef154e/users/8cad68cf07ad434f8c8d415f1c9f8879/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:40.915806 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4248924-2b64-416c-a29d-a5ba90c926a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d24aef0112946b1909ffa96e2ef154e, user_id=8cad68cf07ad434f8c8d415f1c9f8879, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:40.918556 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4248924-2b64-416c-a29d-a5ba90c926a5 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:40.930729 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 966/1940] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:46:40 2022] PUT /identity/v3/projects/3d24aef0112946b1909ffa96e2ef154e/users/8cad68cf07ad434f8c8d415f1c9f8879/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:40.942471 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c9d29-0e9e-4b7a-8d4e-8b419fe30ebb None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:40.942781 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c9d29-0e9e-4b7a-8d4e-8b419fe30ebb None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:40.943847 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c9d29-0e9e-4b7a-8d4e-8b419fe30ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:40.973725 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-146c9d29-0e9e-4b7a-8d4e-8b419fe30ebb None None] MFA Rules not processed for user `8cad68cf07ad434f8c8d415f1c9f8879`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:40.975006 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-146c9d29-0e9e-4b7a-8d4e-8b419fe30ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:40.984809 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:41.054807 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 975/1941] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:40 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:41.066713 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4248924-2b64-416c-a29d-a5ba90c926a5 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:41.069391 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-f4248924-2b64-416c-a29d-a5ba90c926a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:41.139877 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4248924-2b64-416c-a29d-a5ba90c926a5 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:41.144330 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b89803c-2e94-4831-b31b-b24ec6aa9206 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:41.144982 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89803c-2e94-4831-b31b-b24ec6aa9206 service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:41.145289 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89803c-2e94-4831-b31b-b24ec6aa9206 service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:41.145567 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89803c-2e94-4831-b31b-b24ec6aa9206 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:41.155035 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b89803c-2e94-4831-b31b-b24ec6aa9206 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:41.156637 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b89803c-2e94-4831-b31b-b24ec6aa9206 service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:41.167818 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 967/1942] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:46:41 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:46:41.596446 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c9d29-0e9e-4b7a-8d4e-8b419fe30ebb None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:41.659566 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c9d29-0e9e-4b7a-8d4e-8b419fe30ebb None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:41.664579 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e0494-3625-4ba8-b24d-d594d4a596dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:41.665486 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e0494-3625-4ba8-b24d-d594d4a596dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:41.665766 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e0494-3625-4ba8-b24d-d594d4a596dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:41.666046 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e0494-3625-4ba8-b24d-d594d4a596dd admin admin] PATH_INFO: `/v3/users/3ba863143140442abba9ec4d249ee432` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:41.667211 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e0494-3625-4ba8-b24d-d594d4a596dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ba863143140442abba9ec4d249ee432)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:41.669317 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e0494-3625-4ba8-b24d-d594d4a596dd admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:41.721930 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-718e0494-3625-4ba8-b24d-d594d4a596dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ba863143140442abba9ec4d249ee432'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:41.730467 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-718e0494-3625-4ba8-b24d-d594d4a596dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ba863143140442abba9ec4d249ee432'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:41.738425 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-718e0494-3625-4ba8-b24d-d594d4a596dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ba863143140442abba9ec4d249ee432'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:41.739128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-718e0494-3625-4ba8-b24d-d594d4a596dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ba863143140442abba9ec4d249ee432'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:41.752734 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-718e0494-3625-4ba8-b24d-d594d4a596dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ba863143140442abba9ec4d249ee432'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:41.760501 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 976/1943] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:46:41 2022] DELETE /identity/v3/users/3ba863143140442abba9ec4d249ee432 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:41.765215 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6041fd7-a2de-4fce-9338-95883e2ad86e None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:41.765699 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6041fd7-a2de-4fce-9338-95883e2ad86e None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:41.766047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6041fd7-a2de-4fce-9338-95883e2ad86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:41.785387 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-d6041fd7-a2de-4fce-9338-95883e2ad86e None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:41.786580 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-d6041fd7-a2de-4fce-9338-95883e2ad86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:41.858378 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 968/1944] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:41.867548 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e0494-3625-4ba8-b24d-d594d4a596dd admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:41.868957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-718e0494-3625-4ba8-b24d-d594d4a596dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:41.901732 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:41.948483 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e0494-3625-4ba8-b24d-d594d4a596dd admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:41.953836 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e9d89-affc-434d-bb88-7a6e623fa0ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:41.954522 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e9d89-affc-434d-bb88-7a6e623fa0ec service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:41.954794 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e9d89-affc-434d-bb88-7a6e623fa0ec service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:41.955069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e9d89-affc-434d-bb88-7a6e623fa0ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:41.966236 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e9d89-affc-434d-bb88-7a6e623fa0ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:41.968314 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e9d89-affc-434d-bb88-7a6e623fa0ec service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:41.979281 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 977/1945] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:46:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:46:42.268761 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6041fd7-a2de-4fce-9338-95883e2ad86e None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:42.270807 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-d6041fd7-a2de-4fce-9338-95883e2ad86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:42.291220 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6041fd7-a2de-4fce-9338-95883e2ad86e None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:42.301650 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec53fb-f2fc-4d44-a627-0fe1dffb5368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:42.302211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec53fb-f2fc-4d44-a627-0fe1dffb5368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:42.302540 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec53fb-f2fc-4d44-a627-0fe1dffb5368 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:42.302839 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec53fb-f2fc-4d44-a627-0fe1dffb5368 admin admin] PATH_INFO: `/v3/projects/417719f21b7549a2841ab0fa703fa1da` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:42.304212 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec53fb-f2fc-4d44-a627-0fe1dffb5368 admin admin] RBAC: Authorizing `identity:delete_project(project_id=417719f21b7549a2841ab0fa703fa1da)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:42.306257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec53fb-f2fc-4d44-a627-0fe1dffb5368 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:42.308258 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:42.363911 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-75ec53fb-f2fc-4d44-a627-0fe1dffb5368 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '417719f21b7549a2841ab0fa703fa1da', 'request_id': 'req-75ec53fb-f2fc-4d44-a627-0fe1dffb5368'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:42.373202 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-75ec53fb-f2fc-4d44-a627-0fe1dffb5368 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '417719f21b7549a2841ab0fa703fa1da', 'request_id': 'req-75ec53fb-f2fc-4d44-a627-0fe1dffb5368'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:42.381676 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-75ec53fb-f2fc-4d44-a627-0fe1dffb5368 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '417719f21b7549a2841ab0fa703fa1da', 'request_id': 'req-75ec53fb-f2fc-4d44-a627-0fe1dffb5368'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:42.392679 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-75ec53fb-f2fc-4d44-a627-0fe1dffb5368 admin admin] The token cache is being invalidate because project 417719f21b7549a2841ab0fa703fa1da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:46:42.392974 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-75ec53fb-f2fc-4d44-a627-0fe1dffb5368 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:42.393790 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-75ec53fb-f2fc-4d44-a627-0fe1dffb5368 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:42.395415 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 969/1946] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:46:42 2022] DELETE /identity/v3/projects/417719f21b7549a2841ab0fa703fa1da => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:42.400218 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e9d89-affc-434d-bb88-7a6e623fa0ec service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:42.409896 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e9d89-affc-434d-bb88-7a6e623fa0ec service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:42.481294 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0306f1d-ada9-4ef2-ae7b-b3e35f6b34f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:42.482929 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0306f1d-ada9-4ef2-ae7b-b3e35f6b34f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:42.483449 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0306f1d-ada9-4ef2-ae7b-b3e35f6b34f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:42.483888 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0306f1d-ada9-4ef2-ae7b-b3e35f6b34f7 admin admin] PATH_INFO: `/v3/users/8cad68cf07ad434f8c8d415f1c9f8879` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:42.485350 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0306f1d-ada9-4ef2-ae7b-b3e35f6b34f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cad68cf07ad434f8c8d415f1c9f8879)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:42.487539 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0306f1d-ada9-4ef2-ae7b-b3e35f6b34f7 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:42.549468 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-f0306f1d-ada9-4ef2-ae7b-b3e35f6b34f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cad68cf07ad434f8c8d415f1c9f8879'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:42.558146 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-f0306f1d-ada9-4ef2-ae7b-b3e35f6b34f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cad68cf07ad434f8c8d415f1c9f8879'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:42.565295 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-f0306f1d-ada9-4ef2-ae7b-b3e35f6b34f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cad68cf07ad434f8c8d415f1c9f8879'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:42.565933 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-f0306f1d-ada9-4ef2-ae7b-b3e35f6b34f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cad68cf07ad434f8c8d415f1c9f8879'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:42.579343 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-f0306f1d-ada9-4ef2-ae7b-b3e35f6b34f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cad68cf07ad434f8c8d415f1c9f8879'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:42.585988 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 978/1947] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:46:42 2022] DELETE /identity/v3/users/8cad68cf07ad434f8c8d415f1c9f8879 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:42.861423 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec53fb-f2fc-4d44-a627-0fe1dffb5368 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:42.863310 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-75ec53fb-f2fc-4d44-a627-0fe1dffb5368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:42.882104 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec53fb-f2fc-4d44-a627-0fe1dffb5368 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:42.953533 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f1c7e4-9701-4dc1-ac58-6c2eee4531b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:42.954445 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1c7e4-9701-4dc1-ac58-6c2eee4531b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:42.954729 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1c7e4-9701-4dc1-ac58-6c2eee4531b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:42.955007 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1c7e4-9701-4dc1-ac58-6c2eee4531b1 admin admin] PATH_INFO: `/v3/projects/3d24aef0112946b1909ffa96e2ef154e` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:42.956214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f1c7e4-9701-4dc1-ac58-6c2eee4531b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d24aef0112946b1909ffa96e2ef154e)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:42.958219 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f1c7e4-9701-4dc1-ac58-6c2eee4531b1 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:43.018637 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-91f1c7e4-9701-4dc1-ac58-6c2eee4531b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d24aef0112946b1909ffa96e2ef154e', 'request_id': 'req-91f1c7e4-9701-4dc1-ac58-6c2eee4531b1'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:43.027193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-91f1c7e4-9701-4dc1-ac58-6c2eee4531b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d24aef0112946b1909ffa96e2ef154e', 'request_id': 'req-91f1c7e4-9701-4dc1-ac58-6c2eee4531b1'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:43.035272 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-91f1c7e4-9701-4dc1-ac58-6c2eee4531b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d24aef0112946b1909ffa96e2ef154e', 'request_id': 'req-91f1c7e4-9701-4dc1-ac58-6c2eee4531b1'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:43.055993 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-91f1c7e4-9701-4dc1-ac58-6c2eee4531b1 admin admin] The token cache is being invalidate because project 3d24aef0112946b1909ffa96e2ef154e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:46:43.056366 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-91f1c7e4-9701-4dc1-ac58-6c2eee4531b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:43.057110 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-91f1c7e4-9701-4dc1-ac58-6c2eee4531b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:43.058703 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 970/1948] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:46:42 2022] DELETE /identity/v3/projects/3d24aef0112946b1909ffa96e2ef154e => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:43.063546 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fde6d5-2ed4-47ef-847d-401f31eb8e59 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:43.064080 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fde6d5-2ed4-47ef-847d-401f31eb8e59 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:43.064291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fde6d5-2ed4-47ef-847d-401f31eb8e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:43.081887 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-51fde6d5-2ed4-47ef-847d-401f31eb8e59 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:43.083002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-51fde6d5-2ed4-47ef-847d-401f31eb8e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:43.129908 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:43.150293 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 979/1949] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:43.157619 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e9c2b-5f99-46ad-9f36-3cd948c64c42 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:43.157915 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e9c2b-5f99-46ad-9f36-3cd948c64c42 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:43.158216 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e9c2b-5f99-46ad-9f36-3cd948c64c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:43.180220 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-721e9c2b-5f99-46ad-9f36-3cd948c64c42 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:43.181500 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-721e9c2b-5f99-46ad-9f36-3cd948c64c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:43.192325 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 971/1950] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:43.195029 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fde6d5-2ed4-47ef-847d-401f31eb8e59 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:43.205139 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fde6d5-2ed4-47ef-847d-401f31eb8e59 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:43.209853 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17881d1-c091-4800-a1a7-c22f940652c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:43.210518 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17881d1-c091-4800-a1a7-c22f940652c3 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:43.210799 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17881d1-c091-4800-a1a7-c22f940652c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:43.211077 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17881d1-c091-4800-a1a7-c22f940652c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:43.211806 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17881d1-c091-4800-a1a7-c22f940652c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:46:43.212140 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17881d1-c091-4800-a1a7-c22f940652c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:43.213881 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17881d1-c091-4800-a1a7-c22f940652c3 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:43.226368 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 980/1951] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:46:43.230938 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e9c2b-5f99-46ad-9f36-3cd948c64c42 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:43.240853 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e9c2b-5f99-46ad-9f36-3cd948c64c42 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:43.245708 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd471214-916d-4ce0-9069-13a0b74f45ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:43.246285 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd471214-916d-4ce0-9069-13a0b74f45ba admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:43.246595 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd471214-916d-4ce0-9069-13a0b74f45ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:43.246874 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd471214-916d-4ce0-9069-13a0b74f45ba admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:43.247577 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd471214-916d-4ce0-9069-13a0b74f45ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:43.249524 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd471214-916d-4ce0-9069-13a0b74f45ba admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:43.259725 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:43.276882 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 972/1952] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:46:43 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:46:43.280991 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17881d1-c091-4800-a1a7-c22f940652c3 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:43.291544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17881d1-c091-4800-a1a7-c22f940652c3 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:43.351084 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44e140-a5c7-40af-ba3d-2d07e08c7db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:43.351655 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44e140-a5c7-40af-ba3d-2d07e08c7db2 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:43.351938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44e140-a5c7-40af-ba3d-2d07e08c7db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:43.352257 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44e140-a5c7-40af-ba3d-2d07e08c7db2 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:43.352887 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44e140-a5c7-40af-ba3d-2d07e08c7db2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:43.354832 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44e140-a5c7-40af-ba3d-2d07e08c7db2 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:43.377555 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 981/1953] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:46:43 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:46:43.382090 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd471214-916d-4ce0-9069-13a0b74f45ba admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:43.395413 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd471214-916d-4ce0-9069-13a0b74f45ba admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:43.400301 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a2667-5eb2-44f7-9d66-a8159acfc0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:43.400975 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a2667-5eb2-44f7-9d66-a8159acfc0bd admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:43.401288 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a2667-5eb2-44f7-9d66-a8159acfc0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:43.401565 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a2667-5eb2-44f7-9d66-a8159acfc0bd admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:43.402324 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a2667-5eb2-44f7-9d66-a8159acfc0bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:43.404035 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a2667-5eb2-44f7-9d66-a8159acfc0bd admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:43.415355 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 973/1954] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:46:43.419245 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44e140-a5c7-40af-ba3d-2d07e08c7db2 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:43.429583 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44e140-a5c7-40af-ba3d-2d07e08c7db2 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:43.434261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb760c-1cec-4e0c-821a-a97e1bfa222d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:43.434928 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb760c-1cec-4e0c-821a-a97e1bfa222d admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:43.435214 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb760c-1cec-4e0c-821a-a97e1bfa222d admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:43.435515 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb760c-1cec-4e0c-821a-a97e1bfa222d admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:43.436240 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb760c-1cec-4e0c-821a-a97e1bfa222d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:43.437871 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb760c-1cec-4e0c-821a-a97e1bfa222d admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:43.450665 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 982/1955] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:46:43.454967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a2667-5eb2-44f7-9d66-a8159acfc0bd admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:43.465861 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a2667-5eb2-44f7-9d66-a8159acfc0bd admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:43.470629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a0207-920f-48ef-a10b-a1b6d2b4b6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:43.471519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a0207-920f-48ef-a10b-a1b6d2b4b6d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:43.471808 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a0207-920f-48ef-a10b-a1b6d2b4b6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:43.472111 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a0207-920f-48ef-a10b-a1b6d2b4b6d5 admin admin] PATH_INFO: `/v3/projects/79e1ae8953bb4868b213661bd963b596/users/45120da4ce28413d914a29a9fec9ef8a/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:43.491563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a0207-920f-48ef-a10b-a1b6d2b4b6d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79e1ae8953bb4868b213661bd963b596, user_id=45120da4ce28413d914a29a9fec9ef8a, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:43.494302 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a0207-920f-48ef-a10b-a1b6d2b4b6d5 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:43.509348 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 974/1956] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:46:43 2022] PUT /identity/v3/projects/79e1ae8953bb4868b213661bd963b596/users/45120da4ce28413d914a29a9fec9ef8a/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:43.519570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591ae6f-d3ae-4a8f-b00f-9a7931d4f521 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:43.519926 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591ae6f-d3ae-4a8f-b00f-9a7931d4f521 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:43.520292 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591ae6f-d3ae-4a8f-b00f-9a7931d4f521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:43.548981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-0591ae6f-d3ae-4a8f-b00f-9a7931d4f521 None None] MFA Rules not processed for user `45120da4ce28413d914a29a9fec9ef8a`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:43.550369 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-0591ae6f-d3ae-4a8f-b00f-9a7931d4f521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:43.637056 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 983/1957] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:43 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:43.639414 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a0207-920f-48ef-a10b-a1b6d2b4b6d5 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:43.707510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a0207-920f-48ef-a10b-a1b6d2b4b6d5 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:43.713083 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a5648f-7093-41cd-b3a7-59db97ffb37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:43.713746 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5648f-7093-41cd-b3a7-59db97ffb37c admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:43.714034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5648f-7093-41cd-b3a7-59db97ffb37c admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:43.714316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5648f-7093-41cd-b3a7-59db97ffb37c admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:43.715128 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a5648f-7093-41cd-b3a7-59db97ffb37c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:43.717551 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a5648f-7093-41cd-b3a7-59db97ffb37c admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:43.743530 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 975/1958] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:46:43 2022] POST /identity/v3/projects => generated 365 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:46:43.747334 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591ae6f-d3ae-4a8f-b00f-9a7931d4f521 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:43.760439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591ae6f-d3ae-4a8f-b00f-9a7931d4f521 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:43.803193 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:43.819621 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2457e1-ef2f-4897-b01b-7c4fdfe2bb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:43.820293 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2457e1-ef2f-4897-b01b-7c4fdfe2bb03 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:43.820577 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2457e1-ef2f-4897-b01b-7c4fdfe2bb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:43.820851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2457e1-ef2f-4897-b01b-7c4fdfe2bb03 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:43.821587 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2457e1-ef2f-4897-b01b-7c4fdfe2bb03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:43.823485 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2457e1-ef2f-4897-b01b-7c4fdfe2bb03 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:43.843812 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 984/1959] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:46:43 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:46:43.847854 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a5648f-7093-41cd-b3a7-59db97ffb37c admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:43.858644 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a5648f-7093-41cd-b3a7-59db97ffb37c admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:43.863155 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06424d0a-9c79-4d38-9620-71ef1d8732d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:43.863806 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06424d0a-9c79-4d38-9620-71ef1d8732d4 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:43.864087 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06424d0a-9c79-4d38-9620-71ef1d8732d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:43.864362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06424d0a-9c79-4d38-9620-71ef1d8732d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:43.865189 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06424d0a-9c79-4d38-9620-71ef1d8732d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:43.866868 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06424d0a-9c79-4d38-9620-71ef1d8732d4 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:43.879901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 976/1960] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:46:43.881510 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2457e1-ef2f-4897-b01b-7c4fdfe2bb03 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:43.891275 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2457e1-ef2f-4897-b01b-7c4fdfe2bb03 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:43.895879 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bf96b-a636-4d19-9091-3bbf842b4345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:43.897875 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bf96b-a636-4d19-9091-3bbf842b4345 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:43.898161 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bf96b-a636-4d19-9091-3bbf842b4345 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:43.898443 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bf96b-a636-4d19-9091-3bbf842b4345 admin admin] PATH_INFO: `/v3/projects/21b87e92c4a34d39a01344d051f7c14b/users/af2e6f1460e84bdabb3a076e784d7af3/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:43.913883 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19bf96b-a636-4d19-9091-3bbf842b4345 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21b87e92c4a34d39a01344d051f7c14b, user_id=af2e6f1460e84bdabb3a076e784d7af3, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:43.916514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19bf96b-a636-4d19-9091-3bbf842b4345 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:43.929695 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 985/1961] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:46:43 2022] PUT /identity/v3/projects/21b87e92c4a34d39a01344d051f7c14b/users/af2e6f1460e84bdabb3a076e784d7af3/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:43.933626 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06424d0a-9c79-4d38-9620-71ef1d8732d4 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:43.944258 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06424d0a-9c79-4d38-9620-71ef1d8732d4 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:43.998887 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b51b42-af71-49d7-8d3e-4232a83f288b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:43.999551 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b51b42-af71-49d7-8d3e-4232a83f288b admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:43.999832 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b51b42-af71-49d7-8d3e-4232a83f288b admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:44.000104 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b51b42-af71-49d7-8d3e-4232a83f288b admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:44.000827 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b51b42-af71-49d7-8d3e-4232a83f288b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:44.002433 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b51b42-af71-49d7-8d3e-4232a83f288b admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:44.015201 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 977/1962] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:46:44.016356 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bf96b-a636-4d19-9091-3bbf842b4345 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:44.026723 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bf96b-a636-4d19-9091-3bbf842b4345 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:44.031260 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055af54-2bd9-44b0-b92f-bfa152c22c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:44.032126 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055af54-2bd9-44b0-b92f-bfa152c22c91 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:44.032407 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055af54-2bd9-44b0-b92f-bfa152c22c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:44.032684 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055af54-2bd9-44b0-b92f-bfa152c22c91 admin admin] PATH_INFO: `/v3/projects/21b87e92c4a34d39a01344d051f7c14b/users/af2e6f1460e84bdabb3a076e784d7af3/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:44.034676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d055af54-2bd9-44b0-b92f-bfa152c22c91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21b87e92c4a34d39a01344d051f7c14b, user_id=af2e6f1460e84bdabb3a076e784d7af3, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:44.037833 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d055af54-2bd9-44b0-b92f-bfa152c22c91 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:44.048122 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 986/1963] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:46:44 2022] PUT /identity/v3/projects/21b87e92c4a34d39a01344d051f7c14b/users/af2e6f1460e84bdabb3a076e784d7af3/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:44.060120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af671e66-519b-45fc-aead-bc829ba2ad6e None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:44.060411 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af671e66-519b-45fc-aead-bc829ba2ad6e None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:44.060689 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af671e66-519b-45fc-aead-bc829ba2ad6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:44.089046 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-af671e66-519b-45fc-aead-bc829ba2ad6e None None] MFA Rules not processed for user `af2e6f1460e84bdabb3a076e784d7af3`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:44.090273 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-af671e66-519b-45fc-aead-bc829ba2ad6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:44.112944 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:44.183739 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 978/1964] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:44 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:44.201502 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055af54-2bd9-44b0-b92f-bfa152c22c91 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:44.204581 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-d055af54-2bd9-44b0-b92f-bfa152c22c91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:44.286116 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055af54-2bd9-44b0-b92f-bfa152c22c91 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:44.290856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6bf6de-c5be-4b7e-8ccb-e0702ee8a9d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:44.291534 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6bf6de-c5be-4b7e-8ccb-e0702ee8a9d2 service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:44.291823 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6bf6de-c5be-4b7e-8ccb-e0702ee8a9d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:44.292098 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6bf6de-c5be-4b7e-8ccb-e0702ee8a9d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:44.311541 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6bf6de-c5be-4b7e-8ccb-e0702ee8a9d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:44.313131 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6bf6de-c5be-4b7e-8ccb-e0702ee8a9d2 service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:44.325194 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 987/1965] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:46:44 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:46:44.437202 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af671e66-519b-45fc-aead-bc829ba2ad6e None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:44.448464 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af671e66-519b-45fc-aead-bc829ba2ad6e None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:44.453153 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93093907-1051-44aa-a164-846eca099f42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:44.453876 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93093907-1051-44aa-a164-846eca099f42 service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:44.455041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93093907-1051-44aa-a164-846eca099f42 service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:44.455864 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93093907-1051-44aa-a164-846eca099f42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:44.523001 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93093907-1051-44aa-a164-846eca099f42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:44.524841 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93093907-1051-44aa-a164-846eca099f42 service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:44.534918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 979/1966] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:46:44 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:46:54.319870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=50,keystone:INSERT=5 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:54.529984 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=50 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:55.264505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6bf6de-c5be-4b7e-8ccb-e0702ee8a9d2 service cinder] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:55.265935 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-7e6bf6de-c5be-4b7e-8ccb-e0702ee8a9d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:55.363554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6bf6de-c5be-4b7e-8ccb-e0702ee8a9d2 service cinder] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:55.368767 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f665ed4-fbd8-469d-8dfe-fc854f69566f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:55.369675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f665ed4-fbd8-469d-8dfe-fc854f69566f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:55.369963 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f665ed4-fbd8-469d-8dfe-fc854f69566f admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:55.370244 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f665ed4-fbd8-469d-8dfe-fc854f69566f admin admin] PATH_INFO: `/v3/users/b42cf08a07d14ac1a800de7f0ca3517c` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:55.371439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f665ed4-fbd8-469d-8dfe-fc854f69566f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b42cf08a07d14ac1a800de7f0ca3517c)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:55.373392 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f665ed4-fbd8-469d-8dfe-fc854f69566f admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:55.420260 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-3f665ed4-fbd8-469d-8dfe-fc854f69566f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b42cf08a07d14ac1a800de7f0ca3517c'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:55.427516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-3f665ed4-fbd8-469d-8dfe-fc854f69566f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b42cf08a07d14ac1a800de7f0ca3517c'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:55.434944 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-3f665ed4-fbd8-469d-8dfe-fc854f69566f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b42cf08a07d14ac1a800de7f0ca3517c'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:55.435590 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-3f665ed4-fbd8-469d-8dfe-fc854f69566f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b42cf08a07d14ac1a800de7f0ca3517c'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:55.448087 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-3f665ed4-fbd8-469d-8dfe-fc854f69566f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b42cf08a07d14ac1a800de7f0ca3517c'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:55.454081 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 988/1967] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:46:55 2022] DELETE /identity/v3/users/b42cf08a07d14ac1a800de7f0ca3517c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:55.458758 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d328b-24bd-4357-8853-a67bd84b1eb2 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:55.459047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d328b-24bd-4357-8853-a67bd84b1eb2 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:55.459342 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d328b-24bd-4357-8853-a67bd84b1eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:55.477036 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-a81d328b-24bd-4357-8853-a67bd84b1eb2 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:55.478251 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a81d328b-24bd-4357-8853-a67bd84b1eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:55.537315 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 980/1968] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:55.547013 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f665ed4-fbd8-469d-8dfe-fc854f69566f admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:55.548395 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-3f665ed4-fbd8-469d-8dfe-fc854f69566f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:55.608949 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f665ed4-fbd8-469d-8dfe-fc854f69566f admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:55.612991 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1f31c-09d1-4cfe-a0e1-6eadabe290ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:55.613746 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1f31c-09d1-4cfe-a0e1-6eadabe290ad service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:55.614036 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1f31c-09d1-4cfe-a0e1-6eadabe290ad service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:55.614316 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1f31c-09d1-4cfe-a0e1-6eadabe290ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:55.624277 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1f31c-09d1-4cfe-a0e1-6eadabe290ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:55.627875 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1f31c-09d1-4cfe-a0e1-6eadabe290ad service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:55.638694 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 989/1969] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:46:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:46:55.970021 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d328b-24bd-4357-8853-a67bd84b1eb2 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:55.971626 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a81d328b-24bd-4357-8853-a67bd84b1eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:55.985377 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d328b-24bd-4357-8853-a67bd84b1eb2 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:55.991331 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaad8ec-75bb-4333-a444-afe0b0b9aa5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:55.992300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaad8ec-75bb-4333-a444-afe0b0b9aa5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:55.992699 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaad8ec-75bb-4333-a444-afe0b0b9aa5f admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:55.992993 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaad8ec-75bb-4333-a444-afe0b0b9aa5f admin admin] PATH_INFO: `/v3/projects/07ab486bbabf4869b22936fdca66a0c3` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:55.994347 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaad8ec-75bb-4333-a444-afe0b0b9aa5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=07ab486bbabf4869b22936fdca66a0c3)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:55.996516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaad8ec-75bb-4333-a444-afe0b0b9aa5f admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:56.049049 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-ceaad8ec-75bb-4333-a444-afe0b0b9aa5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07ab486bbabf4869b22936fdca66a0c3', 'request_id': 'req-ceaad8ec-75bb-4333-a444-afe0b0b9aa5f'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:56.056678 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-ceaad8ec-75bb-4333-a444-afe0b0b9aa5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07ab486bbabf4869b22936fdca66a0c3', 'request_id': 'req-ceaad8ec-75bb-4333-a444-afe0b0b9aa5f'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:56.064503 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-ceaad8ec-75bb-4333-a444-afe0b0b9aa5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07ab486bbabf4869b22936fdca66a0c3', 'request_id': 'req-ceaad8ec-75bb-4333-a444-afe0b0b9aa5f'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:56.074977 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-ceaad8ec-75bb-4333-a444-afe0b0b9aa5f admin admin] The token cache is being invalidate because project 07ab486bbabf4869b22936fdca66a0c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:46:56.075264 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-ceaad8ec-75bb-4333-a444-afe0b0b9aa5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:56.075881 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-ceaad8ec-75bb-4333-a444-afe0b0b9aa5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:56.077455 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 981/1970] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:46:55 2022] DELETE /identity/v3/projects/07ab486bbabf4869b22936fdca66a0c3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:56.081408 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1f31c-09d1-4cfe-a0e1-6eadabe290ad service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:56.091644 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1f31c-09d1-4cfe-a0e1-6eadabe290ad service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:56.110525 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:56.151378 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae050f29-36c2-4653-8438-389585ece4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:56.152255 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae050f29-36c2-4653-8438-389585ece4a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:56.152552 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae050f29-36c2-4653-8438-389585ece4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:56.152834 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae050f29-36c2-4653-8438-389585ece4a7 admin admin] PATH_INFO: `/v3/users/41bdb637d9514efb893497c619e9a6c4` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:56.154243 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae050f29-36c2-4653-8438-389585ece4a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41bdb637d9514efb893497c619e9a6c4)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:56.156211 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae050f29-36c2-4653-8438-389585ece4a7 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:56.205110 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-ae050f29-36c2-4653-8438-389585ece4a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41bdb637d9514efb893497c619e9a6c4'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:56.213904 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-ae050f29-36c2-4653-8438-389585ece4a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41bdb637d9514efb893497c619e9a6c4'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:56.229596 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-ae050f29-36c2-4653-8438-389585ece4a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41bdb637d9514efb893497c619e9a6c4'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:56.230289 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-ae050f29-36c2-4653-8438-389585ece4a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41bdb637d9514efb893497c619e9a6c4'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:56.242977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-ae050f29-36c2-4653-8438-389585ece4a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41bdb637d9514efb893497c619e9a6c4'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:56.250220 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 990/1971] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:46:56 2022] DELETE /identity/v3/users/41bdb637d9514efb893497c619e9a6c4 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:56.634110 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaad8ec-75bb-4333-a444-afe0b0b9aa5f admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:56.635595 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-ceaad8ec-75bb-4333-a444-afe0b0b9aa5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:56.649443 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaad8ec-75bb-4333-a444-afe0b0b9aa5f admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:56.715858 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e85fc-f5a5-4106-acc5-8cbf51354ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:56.716776 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e85fc-f5a5-4106-acc5-8cbf51354ae7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:56.717093 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e85fc-f5a5-4106-acc5-8cbf51354ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:56.717379 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e85fc-f5a5-4106-acc5-8cbf51354ae7 admin admin] PATH_INFO: `/v3/projects/95cdac43499b49f49cc2e7dd4c7ebac8` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:56.718638 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00e85fc-f5a5-4106-acc5-8cbf51354ae7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95cdac43499b49f49cc2e7dd4c7ebac8)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:56.720570 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00e85fc-f5a5-4106-acc5-8cbf51354ae7 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:56.753345 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:56.773258 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-f00e85fc-f5a5-4106-acc5-8cbf51354ae7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95cdac43499b49f49cc2e7dd4c7ebac8', 'request_id': 'req-f00e85fc-f5a5-4106-acc5-8cbf51354ae7'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:56.779827 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-f00e85fc-f5a5-4106-acc5-8cbf51354ae7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95cdac43499b49f49cc2e7dd4c7ebac8', 'request_id': 'req-f00e85fc-f5a5-4106-acc5-8cbf51354ae7'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:56.787837 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-f00e85fc-f5a5-4106-acc5-8cbf51354ae7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95cdac43499b49f49cc2e7dd4c7ebac8', 'request_id': 'req-f00e85fc-f5a5-4106-acc5-8cbf51354ae7'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:56.799290 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-f00e85fc-f5a5-4106-acc5-8cbf51354ae7 admin admin] The token cache is being invalidate because project 95cdac43499b49f49cc2e7dd4c7ebac8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:46:56.799577 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-f00e85fc-f5a5-4106-acc5-8cbf51354ae7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:56.800241 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-f00e85fc-f5a5-4106-acc5-8cbf51354ae7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:56.802685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 982/1972] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:46:56 2022] DELETE /identity/v3/projects/95cdac43499b49f49cc2e7dd4c7ebac8 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:56.807850 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3413530-c4b7-47d3-87d9-5483447080e6 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:56.808340 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3413530-c4b7-47d3-87d9-5483447080e6 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:56.808429 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3413530-c4b7-47d3-87d9-5483447080e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:56.828564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-c3413530-c4b7-47d3-87d9-5483447080e6 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:56.829732 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-c3413530-c4b7-47d3-87d9-5483447080e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:56.895515 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 991/1973] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:56.904338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abcfd32-172b-45a1-a482-7049ebb25991 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:56.904637 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abcfd32-172b-45a1-a482-7049ebb25991 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:56.904921 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abcfd32-172b-45a1-a482-7049ebb25991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:56.924306 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-5abcfd32-172b-45a1-a482-7049ebb25991 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:56.924924 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-5abcfd32-172b-45a1-a482-7049ebb25991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:56.945599 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 983/1974] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:56.949915 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3413530-c4b7-47d3-87d9-5483447080e6 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:56.965472 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3413530-c4b7-47d3-87d9-5483447080e6 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:56.971465 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05135237-c8b7-4db1-a3a5-47f34aea899e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:56.972157 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05135237-c8b7-4db1-a3a5-47f34aea899e admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:56.972584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05135237-c8b7-4db1-a3a5-47f34aea899e admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:56.972978 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05135237-c8b7-4db1-a3a5-47f34aea899e admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:56.974819 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05135237-c8b7-4db1-a3a5-47f34aea899e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:46:56.974819 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05135237-c8b7-4db1-a3a5-47f34aea899e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:56.976610 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05135237-c8b7-4db1-a3a5-47f34aea899e admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:56.993436 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 992/1975] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:46:56.997753 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abcfd32-172b-45a1-a482-7049ebb25991 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:57.007930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abcfd32-172b-45a1-a482-7049ebb25991 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:57.015525 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5a9d6-477d-4dd3-8b89-096c88e40331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:57.016206 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5a9d6-477d-4dd3-8b89-096c88e40331 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:57.016493 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5a9d6-477d-4dd3-8b89-096c88e40331 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:57.016772 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5a9d6-477d-4dd3-8b89-096c88e40331 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:57.017524 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae5a9d6-477d-4dd3-8b89-096c88e40331 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:57.020605 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae5a9d6-477d-4dd3-8b89-096c88e40331 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:57.050211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 984/1976] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:46:56 2022] POST /identity/v3/projects => generated 381 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:46:57.055408 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05135237-c8b7-4db1-a3a5-47f34aea899e admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:57.070444 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05135237-c8b7-4db1-a3a5-47f34aea899e admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:57.105320 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:57.149824 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cbcf23-fc9b-4e5d-8cba-192ca5e575ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:57.150766 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbcf23-fc9b-4e5d-8cba-192ca5e575ae admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:57.151334 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbcf23-fc9b-4e5d-8cba-192ca5e575ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:57.151769 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbcf23-fc9b-4e5d-8cba-192ca5e575ae admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:57.152890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cbcf23-fc9b-4e5d-8cba-192ca5e575ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:57.154763 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cbcf23-fc9b-4e5d-8cba-192ca5e575ae admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:57.181794 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 993/1977] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:46:57 2022] POST /identity/v3/users => generated 291 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:46:57.188155 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5a9d6-477d-4dd3-8b89-096c88e40331 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:57.202045 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5a9d6-477d-4dd3-8b89-096c88e40331 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:57.212968 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae4710-4b02-475d-a629-9f38357c73ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:57.213773 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae4710-4b02-475d-a629-9f38357c73ea admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:57.214667 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae4710-4b02-475d-a629-9f38357c73ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:57.214910 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae4710-4b02-475d-a629-9f38357c73ea admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:57.215866 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ae4710-4b02-475d-a629-9f38357c73ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:57.217965 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ae4710-4b02-475d-a629-9f38357c73ea admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:57.233545 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 985/1978] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:46:57.237865 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cbcf23-fc9b-4e5d-8cba-192ca5e575ae admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:57.250131 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cbcf23-fc9b-4e5d-8cba-192ca5e575ae admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:57.256193 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b0bd19-d869-450d-af41-0b64cd634410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:57.257646 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0bd19-d869-450d-af41-0b64cd634410 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:57.258143 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0bd19-d869-450d-af41-0b64cd634410 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:57.258497 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0bd19-d869-450d-af41-0b64cd634410 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:57.259435 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b0bd19-d869-450d-af41-0b64cd634410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:57.261575 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b0bd19-d869-450d-af41-0b64cd634410 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:57.272948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 994/1979] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:46:57.276964 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae4710-4b02-475d-a629-9f38357c73ea admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:57.287844 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae4710-4b02-475d-a629-9f38357c73ea admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:57.292666 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd6ab7-f401-4917-9b76-a5ae05a72fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:57.293670 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd6ab7-f401-4917-9b76-a5ae05a72fc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:57.294733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd6ab7-f401-4917-9b76-a5ae05a72fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:57.294733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd6ab7-f401-4917-9b76-a5ae05a72fc3 admin admin] PATH_INFO: `/v3/projects/122f2c5206a74b25b3ce8ad6f9727e7a/users/8a092bdf22654fcb9e669a9dc2065558/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:57.314042 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd6ab7-f401-4917-9b76-a5ae05a72fc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=122f2c5206a74b25b3ce8ad6f9727e7a, user_id=8a092bdf22654fcb9e669a9dc2065558, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:57.316743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd6ab7-f401-4917-9b76-a5ae05a72fc3 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:57.345219 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 986/1980] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:46:57 2022] PUT /identity/v3/projects/122f2c5206a74b25b3ce8ad6f9727e7a/users/8a092bdf22654fcb9e669a9dc2065558/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:57.356243 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc099de-5292-429b-9a6f-7750df00bbf0 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:57.356531 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc099de-5292-429b-9a6f-7750df00bbf0 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:57.356810 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc099de-5292-429b-9a6f-7750df00bbf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:57.393606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-cfc099de-5292-429b-9a6f-7750df00bbf0 None None] MFA Rules not processed for user `8a092bdf22654fcb9e669a9dc2065558`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:57.394827 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-cfc099de-5292-429b-9a6f-7750df00bbf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:57.472877 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 995/1981] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:57.478436 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd6ab7-f401-4917-9b76-a5ae05a72fc3 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:57.545635 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd6ab7-f401-4917-9b76-a5ae05a72fc3 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:57.550335 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a49286-db9e-4042-b0a1-d06f4779054c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:57.551254 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a49286-db9e-4042-b0a1-d06f4779054c admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:57.551532 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a49286-db9e-4042-b0a1-d06f4779054c admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:57.551835 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a49286-db9e-4042-b0a1-d06f4779054c admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:57.552697 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a49286-db9e-4042-b0a1-d06f4779054c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:57.554635 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a49286-db9e-4042-b0a1-d06f4779054c admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:57.587764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 987/1982] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:46:57 2022] POST /identity/v3/projects => generated 381 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:46:57.590901 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc099de-5292-429b-9a6f-7750df00bbf0 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:57.607624 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc099de-5292-429b-9a6f-7750df00bbf0 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:57.684188 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c5aa7-0e79-4d15-8d3f-f7e7387f228b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:57.684870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c5aa7-0e79-4d15-8d3f-f7e7387f228b admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:57.685253 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c5aa7-0e79-4d15-8d3f-f7e7387f228b admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:57.685509 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c5aa7-0e79-4d15-8d3f-f7e7387f228b admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:57.686158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5c5aa7-0e79-4d15-8d3f-f7e7387f228b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:57.688090 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5c5aa7-0e79-4d15-8d3f-f7e7387f228b admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:57.724267 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 996/1983] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:46:57 2022] POST /identity/v3/users => generated 297 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:46:57.727974 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a49286-db9e-4042-b0a1-d06f4779054c admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:57.738508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a49286-db9e-4042-b0a1-d06f4779054c admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:57.743389 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d291d-f241-49b4-b120-20419c07f2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:57.744061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d291d-f241-49b4-b120-20419c07f2b2 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:57.744349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d291d-f241-49b4-b120-20419c07f2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:57.744629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d291d-f241-49b4-b120-20419c07f2b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:57.745392 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d291d-f241-49b4-b120-20419c07f2b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:57.746997 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d291d-f241-49b4-b120-20419c07f2b2 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:57.758814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 988/1984] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:46:57.763029 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c5aa7-0e79-4d15-8d3f-f7e7387f228b admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:57.774044 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c5aa7-0e79-4d15-8d3f-f7e7387f228b admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:57.778671 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e856c-bae1-4ce3-b00a-b73cc3a20ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:57.779559 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e856c-bae1-4ce3-b00a-b73cc3a20ee4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:57.779847 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e856c-bae1-4ce3-b00a-b73cc3a20ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:57.780130 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e856c-bae1-4ce3-b00a-b73cc3a20ee4 admin admin] PATH_INFO: `/v3/projects/3717537d0c584ec18919b5039ce2c8ab/users/347ba4876fa44bf190ef51ed9f8d7c53/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:57.796217 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e856c-bae1-4ce3-b00a-b73cc3a20ee4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3717537d0c584ec18919b5039ce2c8ab, user_id=347ba4876fa44bf190ef51ed9f8d7c53, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:57.798899 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e856c-bae1-4ce3-b00a-b73cc3a20ee4 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:57.812201 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 997/1985] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:46:57 2022] PUT /identity/v3/projects/3717537d0c584ec18919b5039ce2c8ab/users/347ba4876fa44bf190ef51ed9f8d7c53/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:57.816126 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d291d-f241-49b4-b120-20419c07f2b2 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:57.826081 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d291d-f241-49b4-b120-20419c07f2b2 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:57.847326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:57.889081 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0350708c-a6f3-4be4-b683-cf104b2a1e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:57.889656 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350708c-a6f3-4be4-b683-cf104b2a1e78 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:57.889956 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350708c-a6f3-4be4-b683-cf104b2a1e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:57.890253 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350708c-a6f3-4be4-b683-cf104b2a1e78 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:57.890970 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0350708c-a6f3-4be4-b683-cf104b2a1e78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:57.892586 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0350708c-a6f3-4be4-b683-cf104b2a1e78 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:57.902237 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 989/1986] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:46:57.906353 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e856c-bae1-4ce3-b00a-b73cc3a20ee4 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:57.912228 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:57.918381 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e856c-bae1-4ce3-b00a-b73cc3a20ee4 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:57.923398 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f185037-2ad3-42b1-81f4-f4602b166e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:57.924329 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f185037-2ad3-42b1-81f4-f4602b166e78 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:57.924609 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f185037-2ad3-42b1-81f4-f4602b166e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:57.924882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f185037-2ad3-42b1-81f4-f4602b166e78 admin admin] PATH_INFO: `/v3/projects/3717537d0c584ec18919b5039ce2c8ab/users/347ba4876fa44bf190ef51ed9f8d7c53/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:57.927803 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f185037-2ad3-42b1-81f4-f4602b166e78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3717537d0c584ec18919b5039ce2c8ab, user_id=347ba4876fa44bf190ef51ed9f8d7c53, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:57.930483 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f185037-2ad3-42b1-81f4-f4602b166e78 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:57.940567 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 998/1987] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:46:57 2022] PUT /identity/v3/projects/3717537d0c584ec18919b5039ce2c8ab/users/347ba4876fa44bf190ef51ed9f8d7c53/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:57.950805 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7903a-8b3e-4ccf-aa83-bc3ec31d1b66 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:57.951090 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7903a-8b3e-4ccf-aa83-bc3ec31d1b66 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:57.951366 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7903a-8b3e-4ccf-aa83-bc3ec31d1b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:57.977126 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-f7c7903a-8b3e-4ccf-aa83-bc3ec31d1b66 None None] MFA Rules not processed for user `347ba4876fa44bf190ef51ed9f8d7c53`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:57.978326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-f7c7903a-8b3e-4ccf-aa83-bc3ec31d1b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:58.046928 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 990/1988] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:58.060348 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f185037-2ad3-42b1-81f4-f4602b166e78 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:58.062052 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-7f185037-2ad3-42b1-81f4-f4602b166e78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:58.118966 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f185037-2ad3-42b1-81f4-f4602b166e78 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:58.122527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48489b8-d345-420f-9a38-6d8343c4f72e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:58.123189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48489b8-d345-420f-9a38-6d8343c4f72e service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:58.123469 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48489b8-d345-420f-9a38-6d8343c4f72e service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:58.123745 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48489b8-d345-420f-9a38-6d8343c4f72e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:58.132661 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48489b8-d345-420f-9a38-6d8343c4f72e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:58.134103 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48489b8-d345-420f-9a38-6d8343c4f72e service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:58.143332 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 999/1989] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:46:58 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:46:58.493109 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7903a-8b3e-4ccf-aa83-bc3ec31d1b66 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:58.495940 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-f7c7903a-8b3e-4ccf-aa83-bc3ec31d1b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:58.526424 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48489b8-d345-420f-9a38-6d8343c4f72e service cinder] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:58.538010 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48489b8-d345-420f-9a38-6d8343c4f72e service cinder] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:58.542578 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa869b-4a37-42df-aa92-712156ab8a34 tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '347ba4876fa44bf190ef51ed9f8d7c53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3717537d0c584ec18919b5039ce2c8ab', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:58.543239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa869b-4a37-42df-aa92-712156ab8a34 tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:58.543547 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa869b-4a37-42df-aa92-712156ab8a34 tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:58.544066 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa869b-4a37-42df-aa92-712156ab8a34 tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:58.544832 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfa869b-4a37-42df-aa92-712156ab8a34 tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:46:58.545169 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfa869b-4a37-42df-aa92-712156ab8a34 tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:58.546749 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfa869b-4a37-42df-aa92-712156ab8a34 tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:58.560341 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1000/1990] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:46:58.564326 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa869b-4a37-42df-aa92-712156ab8a34 tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:58.576410 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa869b-4a37-42df-aa92-712156ab8a34 tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:58.582196 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455f005-3734-42ab-87c5-bee5baff94c6 tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '347ba4876fa44bf190ef51ed9f8d7c53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3717537d0c584ec18919b5039ce2c8ab', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:58.582767 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7903a-8b3e-4ccf-aa83-bc3ec31d1b66 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:58.583044 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455f005-3734-42ab-87c5-bee5baff94c6 tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:58.583531 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455f005-3734-42ab-87c5-bee5baff94c6 tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:58.583856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455f005-3734-42ab-87c5-bee5baff94c6 tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:58.584564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3455f005-3734-42ab-87c5-bee5baff94c6 tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:58.586505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3455f005-3734-42ab-87c5-bee5baff94c6 tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:58.588294 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26301045-7cc4-493b-8991-9a6a04405700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:58.589440 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26301045-7cc4-493b-8991-9a6a04405700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:58.589757 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26301045-7cc4-493b-8991-9a6a04405700 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:58.590118 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26301045-7cc4-493b-8991-9a6a04405700 admin admin] PATH_INFO: `/v3/users/45120da4ce28413d914a29a9fec9ef8a` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:58.591805 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26301045-7cc4-493b-8991-9a6a04405700 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45120da4ce28413d914a29a9fec9ef8a)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:58.594311 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26301045-7cc4-493b-8991-9a6a04405700 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:58.624150 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1001/1991] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:46:58 2022] POST /identity/v3/projects => generated 355 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:46:58.628246 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455f005-3734-42ab-87c5-bee5baff94c6 tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:58.654295 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-26301045-7cc4-493b-8991-9a6a04405700 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45120da4ce28413d914a29a9fec9ef8a'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:58.655903 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:58.663232 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-26301045-7cc4-493b-8991-9a6a04405700 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45120da4ce28413d914a29a9fec9ef8a'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:58.663870 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-26301045-7cc4-493b-8991-9a6a04405700 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45120da4ce28413d914a29a9fec9ef8a'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:58.672714 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-26301045-7cc4-493b-8991-9a6a04405700 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45120da4ce28413d914a29a9fec9ef8a'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:58.687310 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-26301045-7cc4-493b-8991-9a6a04405700 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45120da4ce28413d914a29a9fec9ef8a'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:58.697563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 991/1992] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:46:58 2022] DELETE /identity/v3/users/45120da4ce28413d914a29a9fec9ef8a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:58.699028 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81d931-1dda-448f-bc5a-4e9e47994d35 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:58.699323 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81d931-1dda-448f-bc5a-4e9e47994d35 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:58.699607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81d931-1dda-448f-bc5a-4e9e47994d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:58.708951 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455f005-3734-42ab-87c5-bee5baff94c6 tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:58.718591 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-ee81d931-1dda-448f-bc5a-4e9e47994d35 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:58.719811 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-ee81d931-1dda-448f-bc5a-4e9e47994d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:58.781912 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff179dbf-df37-46a4-8ef1-d015580bf95c tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '347ba4876fa44bf190ef51ed9f8d7c53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3717537d0c584ec18919b5039ce2c8ab', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:58.782851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff179dbf-df37-46a4-8ef1-d015580bf95c tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:58.783294 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff179dbf-df37-46a4-8ef1-d015580bf95c tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:58.783571 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff179dbf-df37-46a4-8ef1-d015580bf95c tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:58.784302 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff179dbf-df37-46a4-8ef1-d015580bf95c tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:46:58.784610 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff179dbf-df37-46a4-8ef1-d015580bf95c tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:58.786788 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff179dbf-df37-46a4-8ef1-d015580bf95c tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:58.789796 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 992/1993] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:58.793728 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:58.799502 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1002/1994] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:46:58.801191 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81d931-1dda-448f-bc5a-4e9e47994d35 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:58.804761 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-ee81d931-1dda-448f-bc5a-4e9e47994d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:58.846373 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff179dbf-df37-46a4-8ef1-d015580bf95c tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:58.848385 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-ff179dbf-df37-46a4-8ef1-d015580bf95c tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:58.864280 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff179dbf-df37-46a4-8ef1-d015580bf95c tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:58.871030 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4588b53-26a8-4ca3-bd7c-5f29f283ae9b tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '347ba4876fa44bf190ef51ed9f8d7c53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3717537d0c584ec18919b5039ce2c8ab', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:58.871865 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4588b53-26a8-4ca3-bd7c-5f29f283ae9b tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:58.872161 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4588b53-26a8-4ca3-bd7c-5f29f283ae9b tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:58.873037 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4588b53-26a8-4ca3-bd7c-5f29f283ae9b tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] PATH_INFO: `/v3/projects/517004b30a09428fac48a016692504c7` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:58.873718 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4588b53-26a8-4ca3-bd7c-5f29f283ae9b tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] RBAC: Authorizing `identity:delete_project(project_id=517004b30a09428fac48a016692504c7)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:58.875774 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4588b53-26a8-4ca3-bd7c-5f29f283ae9b tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:58.876791 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81d931-1dda-448f-bc5a-4e9e47994d35 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:58.882382 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dbbb73-b4cf-47cc-b13e-1d98f31d5892 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:58.883341 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbbb73-b4cf-47cc-b13e-1d98f31d5892 service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:58.883626 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbbb73-b4cf-47cc-b13e-1d98f31d5892 service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:58.884030 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbbb73-b4cf-47cc-b13e-1d98f31d5892 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:58.895135 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dbbb73-b4cf-47cc-b13e-1d98f31d5892 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:58.896769 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dbbb73-b4cf-47cc-b13e-1d98f31d5892 service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:58.909277 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 993/1995] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:46:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 13:46:58.943990 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c4588b53-26a8-4ca3-bd7c-5f29f283ae9b tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '517004b30a09428fac48a016692504c7', 'request_id': 'req-c4588b53-26a8-4ca3-bd7c-5f29f283ae9b'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:58.952362 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c4588b53-26a8-4ca3-bd7c-5f29f283ae9b tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '517004b30a09428fac48a016692504c7', 'request_id': 'req-c4588b53-26a8-4ca3-bd7c-5f29f283ae9b'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:58.962473 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c4588b53-26a8-4ca3-bd7c-5f29f283ae9b tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '517004b30a09428fac48a016692504c7', 'request_id': 'req-c4588b53-26a8-4ca3-bd7c-5f29f283ae9b'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:58.976344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c4588b53-26a8-4ca3-bd7c-5f29f283ae9b tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] The token cache is being invalidate because project 517004b30a09428fac48a016692504c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:46:58.976592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c4588b53-26a8-4ca3-bd7c-5f29f283ae9b tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:58.977214 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c4588b53-26a8-4ca3-bd7c-5f29f283ae9b tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:58.979223 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1003/1996] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:46:58 2022] DELETE /identity/v3/projects/517004b30a09428fac48a016692504c7 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:46:59.219224 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dbbb73-b4cf-47cc-b13e-1d98f31d5892 service neutron] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:59.221512 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-16dbbb73-b4cf-47cc-b13e-1d98f31d5892 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:59.237992 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dbbb73-b4cf-47cc-b13e-1d98f31d5892 service neutron] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:59.313538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05a839-85ff-4ee7-b86f-2576134871ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:59.314500 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05a839-85ff-4ee7-b86f-2576134871ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:59.315120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05a839-85ff-4ee7-b86f-2576134871ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:59.315468 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05a839-85ff-4ee7-b86f-2576134871ba admin admin] PATH_INFO: `/v3/users/8a092bdf22654fcb9e669a9dc2065558` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:59.316952 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd05a839-85ff-4ee7-b86f-2576134871ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a092bdf22654fcb9e669a9dc2065558)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:59.320331 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd05a839-85ff-4ee7-b86f-2576134871ba admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:59.321626 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4588b53-26a8-4ca3-bd7c-5f29f283ae9b tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:59.324607 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-c4588b53-26a8-4ca3-bd7c-5f29f283ae9b tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:59.346044 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4588b53-26a8-4ca3-bd7c-5f29f283ae9b tempest-VolumeQuotaClassesTest-1380775726 tempest-VolumeQuotaClassesTest-1380775726-project-admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:59.351916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:59.355379 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef88c32-4ffa-477e-bebc-87b4192babaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:59.356498 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef88c32-4ffa-477e-bebc-87b4192babaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:59.356985 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef88c32-4ffa-477e-bebc-87b4192babaa admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:59.357518 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef88c32-4ffa-477e-bebc-87b4192babaa admin admin] PATH_INFO: `/v3/projects/79e1ae8953bb4868b213661bd963b596` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:59.359012 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef88c32-4ffa-477e-bebc-87b4192babaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=79e1ae8953bb4868b213661bd963b596)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:59.361136 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef88c32-4ffa-477e-bebc-87b4192babaa admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:59.410516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-fd05a839-85ff-4ee7-b86f-2576134871ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a092bdf22654fcb9e669a9dc2065558'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:59.422195 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-fd05a839-85ff-4ee7-b86f-2576134871ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a092bdf22654fcb9e669a9dc2065558'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:59.423429 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-fd05a839-85ff-4ee7-b86f-2576134871ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a092bdf22654fcb9e669a9dc2065558'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:59.432444 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-1ef88c32-4ffa-477e-bebc-87b4192babaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79e1ae8953bb4868b213661bd963b596', 'request_id': 'req-1ef88c32-4ffa-477e-bebc-87b4192babaa'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:59.432699 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-fd05a839-85ff-4ee7-b86f-2576134871ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a092bdf22654fcb9e669a9dc2065558'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:59.440369 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-1ef88c32-4ffa-477e-bebc-87b4192babaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79e1ae8953bb4868b213661bd963b596', 'request_id': 'req-1ef88c32-4ffa-477e-bebc-87b4192babaa'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:59.449227 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-fd05a839-85ff-4ee7-b86f-2576134871ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a092bdf22654fcb9e669a9dc2065558'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:59.451183 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-1ef88c32-4ffa-477e-bebc-87b4192babaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79e1ae8953bb4868b213661bd963b596', 'request_id': 'req-1ef88c32-4ffa-477e-bebc-87b4192babaa'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:59.461042 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 994/1997] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:46:59 2022] DELETE /identity/v3/users/8a092bdf22654fcb9e669a9dc2065558 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:59.464722 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4a076-6c50-4948-8c09-553050d459a3 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:59.465340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4a076-6c50-4948-8c09-553050d459a3 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:59.466163 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4a076-6c50-4948-8c09-553050d459a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:59.468770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-1ef88c32-4ffa-477e-bebc-87b4192babaa admin admin] The token cache is being invalidate because project 79e1ae8953bb4868b213661bd963b596 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:46:59.470052 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-1ef88c32-4ffa-477e-bebc-87b4192babaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:59.473220 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-1ef88c32-4ffa-477e-bebc-87b4192babaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:59.480524 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1004/1998] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:46:59 2022] DELETE /identity/v3/projects/79e1ae8953bb4868b213661bd963b596 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:46:59.492043 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-28d4a076-6c50-4948-8c09-553050d459a3 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:46:59.492043 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-28d4a076-6c50-4948-8c09-553050d459a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:59.494387 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef88c32-4ffa-477e-bebc-87b4192babaa admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:59.495812 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-1ef88c32-4ffa-477e-bebc-87b4192babaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:59.572441 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 995/1999] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:46:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:46:59.581084 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4a076-6c50-4948-8c09-553050d459a3 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:46:59.582670 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-28d4a076-6c50-4948-8c09-553050d459a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:46:59.583798 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef88c32-4ffa-477e-bebc-87b4192babaa admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:59.593411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec479480-00a9-479c-aabb-113f86074c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:59.594588 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec479480-00a9-479c-aabb-113f86074c4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:59.595081 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec479480-00a9-479c-aabb-113f86074c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:59.595676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec479480-00a9-479c-aabb-113f86074c4a admin admin] PATH_INFO: `/v3/users/af2e6f1460e84bdabb3a076e784d7af3` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:59.597922 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec479480-00a9-479c-aabb-113f86074c4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=af2e6f1460e84bdabb3a076e784d7af3)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:59.600110 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec479480-00a9-479c-aabb-113f86074c4a admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:59.606527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:46:59.659273 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4a076-6c50-4948-8c09-553050d459a3 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:46:59.671776 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60cd3b9-4cfc-41fb-bdef-12deb9084115 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:46:59.673663 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60cd3b9-4cfc-41fb-bdef-12deb9084115 service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:46:59.677556 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60cd3b9-4cfc-41fb-bdef-12deb9084115 service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:46:59.677864 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60cd3b9-4cfc-41fb-bdef-12deb9084115 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:46:59.690411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-ec479480-00a9-479c-aabb-113f86074c4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af2e6f1460e84bdabb3a076e784d7af3'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:59.693365 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60cd3b9-4cfc-41fb-bdef-12deb9084115 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:46:59.695474 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60cd3b9-4cfc-41fb-bdef-12deb9084115 service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:46:59.701499 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-ec479480-00a9-479c-aabb-113f86074c4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af2e6f1460e84bdabb3a076e784d7af3'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:59.708685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 996/2000] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:46:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 13:46:59.713109 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-ec479480-00a9-479c-aabb-113f86074c4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af2e6f1460e84bdabb3a076e784d7af3'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:59.714189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-ec479480-00a9-479c-aabb-113f86074c4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af2e6f1460e84bdabb3a076e784d7af3'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:59.730623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-ec479480-00a9-479c-aabb-113f86074c4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af2e6f1460e84bdabb3a076e784d7af3'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:46:59.738829 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1005/2001] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:46:59 2022] DELETE /identity/v3/users/af2e6f1460e84bdabb3a076e784d7af3 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:00.127184 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60cd3b9-4cfc-41fb-bdef-12deb9084115 service neutron] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:00.129274 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-f60cd3b9-4cfc-41fb-bdef-12deb9084115 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:00.150622 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60cd3b9-4cfc-41fb-bdef-12deb9084115 service neutron] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:00.229928 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:00.263667 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d5b9f-2e1d-4748-8c13-bb2bae12fc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:00.264723 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d5b9f-2e1d-4748-8c13-bb2bae12fc80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:00.264964 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d5b9f-2e1d-4748-8c13-bb2bae12fc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:00.265233 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d5b9f-2e1d-4748-8c13-bb2bae12fc80 admin admin] PATH_INFO: `/v3/projects/122f2c5206a74b25b3ce8ad6f9727e7a` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:00.266673 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d5b9f-2e1d-4748-8c13-bb2bae12fc80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=122f2c5206a74b25b3ce8ad6f9727e7a)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:00.269680 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d5b9f-2e1d-4748-8c13-bb2bae12fc80 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:00.344130 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-026d5b9f-2e1d-4748-8c13-bb2bae12fc80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '122f2c5206a74b25b3ce8ad6f9727e7a', 'request_id': 'req-026d5b9f-2e1d-4748-8c13-bb2bae12fc80'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:00.354468 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-026d5b9f-2e1d-4748-8c13-bb2bae12fc80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '122f2c5206a74b25b3ce8ad6f9727e7a', 'request_id': 'req-026d5b9f-2e1d-4748-8c13-bb2bae12fc80'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:00.365633 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-026d5b9f-2e1d-4748-8c13-bb2bae12fc80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '122f2c5206a74b25b3ce8ad6f9727e7a', 'request_id': 'req-026d5b9f-2e1d-4748-8c13-bb2bae12fc80'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:00.386830 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-026d5b9f-2e1d-4748-8c13-bb2bae12fc80 admin admin] The token cache is being invalidate because project 122f2c5206a74b25b3ce8ad6f9727e7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:47:00.387543 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-026d5b9f-2e1d-4748-8c13-bb2bae12fc80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:00.389689 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-026d5b9f-2e1d-4748-8c13-bb2bae12fc80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:00.392995 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 997/2002] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:47:00 2022] DELETE /identity/v3/projects/122f2c5206a74b25b3ce8ad6f9727e7a => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:00.397095 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec479480-00a9-479c-aabb-113f86074c4a admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:00.398531 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-ec479480-00a9-479c-aabb-113f86074c4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:00.412109 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec479480-00a9-479c-aabb-113f86074c4a admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:00.445125 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d5b9f-2e1d-4748-8c13-bb2bae12fc80 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:00.446742 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-026d5b9f-2e1d-4748-8c13-bb2bae12fc80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:00.569874 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f229a-888f-4e61-85a3-d7f517275680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:00.570924 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f229a-888f-4e61-85a3-d7f517275680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:00.571311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f229a-888f-4e61-85a3-d7f517275680 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:00.571697 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f229a-888f-4e61-85a3-d7f517275680 admin admin] PATH_INFO: `/v3/users/347ba4876fa44bf190ef51ed9f8d7c53` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:00.571884 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d5b9f-2e1d-4748-8c13-bb2bae12fc80 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:00.573294 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f229a-888f-4e61-85a3-d7f517275680 admin admin] RBAC: Authorizing `identity:delete_user(user_id=347ba4876fa44bf190ef51ed9f8d7c53)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:00.575297 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f229a-888f-4e61-85a3-d7f517275680 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:00.577268 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ea1c9-089d-46e8-86e8-b161babccb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:00.578759 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ea1c9-089d-46e8-86e8-b161babccb7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:00.579375 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ea1c9-089d-46e8-86e8-b161babccb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:00.579732 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ea1c9-089d-46e8-86e8-b161babccb7c admin admin] PATH_INFO: `/v3/projects/21b87e92c4a34d39a01344d051f7c14b` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:00.581959 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ea1c9-089d-46e8-86e8-b161babccb7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=21b87e92c4a34d39a01344d051f7c14b)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:00.585335 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ea1c9-089d-46e8-86e8-b161babccb7c admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:00.638075 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-673f229a-888f-4e61-85a3-d7f517275680 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '347ba4876fa44bf190ef51ed9f8d7c53'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:00.649728 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-673f229a-888f-4e61-85a3-d7f517275680 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '347ba4876fa44bf190ef51ed9f8d7c53'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:00.665760 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-673f229a-888f-4e61-85a3-d7f517275680 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '347ba4876fa44bf190ef51ed9f8d7c53'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:00.668274 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-673f229a-888f-4e61-85a3-d7f517275680 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '347ba4876fa44bf190ef51ed9f8d7c53'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:00.695370 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-957ea1c9-089d-46e8-86e8-b161babccb7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21b87e92c4a34d39a01344d051f7c14b', 'request_id': 'req-957ea1c9-089d-46e8-86e8-b161babccb7c'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:00.704302 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-957ea1c9-089d-46e8-86e8-b161babccb7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21b87e92c4a34d39a01344d051f7c14b', 'request_id': 'req-957ea1c9-089d-46e8-86e8-b161babccb7c'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:00.709388 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-673f229a-888f-4e61-85a3-d7f517275680 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '347ba4876fa44bf190ef51ed9f8d7c53'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:00.719225 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-957ea1c9-089d-46e8-86e8-b161babccb7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21b87e92c4a34d39a01344d051f7c14b', 'request_id': 'req-957ea1c9-089d-46e8-86e8-b161babccb7c'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:00.725260 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:00.734379 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1006/2003] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:47:00 2022] DELETE /identity/v3/users/347ba4876fa44bf190ef51ed9f8d7c53 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:00.744613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-957ea1c9-089d-46e8-86e8-b161babccb7c admin admin] The token cache is being invalidate because project 21b87e92c4a34d39a01344d051f7c14b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:47:00.745087 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-957ea1c9-089d-46e8-86e8-b161babccb7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:00.746010 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-957ea1c9-089d-46e8-86e8-b161babccb7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:00.747985 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 998/2004] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:47:00 2022] DELETE /identity/v3/projects/21b87e92c4a34d39a01344d051f7c14b => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:47:00.755675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8694d5-37b5-4081-9856-498ca78ee5b1 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:00.755675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8694d5-37b5-4081-9856-498ca78ee5b1 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:00.756008 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8694d5-37b5-4081-9856-498ca78ee5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:00.767046 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:00.787320 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-ef8694d5-37b5-4081-9856-498ca78ee5b1 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:00.788352 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-ef8694d5-37b5-4081-9856-498ca78ee5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:00.869930 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1007/2005] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:00.877759 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3880719-18a7-44ae-b054-2a3dd2055714 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:00.878055 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3880719-18a7-44ae-b054-2a3dd2055714 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:00.878337 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3880719-18a7-44ae-b054-2a3dd2055714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:00.899419 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-f3880719-18a7-44ae-b054-2a3dd2055714 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:00.900679 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-f3880719-18a7-44ae-b054-2a3dd2055714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:00.909484 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 999/2006] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:00.914657 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8694d5-37b5-4081-9856-498ca78ee5b1 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:00.934279 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8694d5-37b5-4081-9856-498ca78ee5b1 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:00.942408 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d438e2-89bb-42a3-bdbf-1f05557ab233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:00.943084 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d438e2-89bb-42a3-bdbf-1f05557ab233 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:00.943654 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d438e2-89bb-42a3-bdbf-1f05557ab233 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:00.944730 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d438e2-89bb-42a3-bdbf-1f05557ab233 admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:00.945580 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d438e2-89bb-42a3-bdbf-1f05557ab233 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:47:00.945955 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d438e2-89bb-42a3-bdbf-1f05557ab233 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:00.947638 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d438e2-89bb-42a3-bdbf-1f05557ab233 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:00.982417 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1008/2007] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:47:00.986363 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3880719-18a7-44ae-b054-2a3dd2055714 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:00.998957 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3880719-18a7-44ae-b054-2a3dd2055714 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:01.004809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d964954-26cb-451d-9cf2-25c2fdbe65bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:01.012098 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d964954-26cb-451d-9cf2-25c2fdbe65bc admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:01.012937 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d964954-26cb-451d-9cf2-25c2fdbe65bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:01.012937 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d964954-26cb-451d-9cf2-25c2fdbe65bc admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:01.013440 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d964954-26cb-451d-9cf2-25c2fdbe65bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:01.015386 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d964954-26cb-451d-9cf2-25c2fdbe65bc admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:01.062562 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1000/2008] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:47:00 2022] POST /identity/v3/projects => generated 385 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:01.063651 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d438e2-89bb-42a3-bdbf-1f05557ab233 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:01.076657 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d438e2-89bb-42a3-bdbf-1f05557ab233 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:01.151680 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d0332-0d3d-44de-8fff-041036607ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:01.152745 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d0332-0d3d-44de-8fff-041036607ca6 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:01.153311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d0332-0d3d-44de-8fff-041036607ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:01.153781 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d0332-0d3d-44de-8fff-041036607ca6 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:01.154734 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2d0332-0d3d-44de-8fff-041036607ca6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:01.156934 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2d0332-0d3d-44de-8fff-041036607ca6 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:01.191783 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1009/2009] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:47:01 2022] POST /identity/v3/users => generated 293 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:01.193989 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d964954-26cb-451d-9cf2-25c2fdbe65bc admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:01.207061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d964954-26cb-451d-9cf2-25c2fdbe65bc admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:01.222988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69f092a-af67-407d-bd77-c63b9409bcac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:01.224241 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f092a-af67-407d-bd77-c63b9409bcac admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:01.225218 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f092a-af67-407d-bd77-c63b9409bcac admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:01.225492 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f092a-af67-407d-bd77-c63b9409bcac admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:01.226252 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69f092a-af67-407d-bd77-c63b9409bcac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:01.227890 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69f092a-af67-407d-bd77-c63b9409bcac admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:01.241154 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1001/2010] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:47:01.245383 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d0332-0d3d-44de-8fff-041036607ca6 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:01.259622 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d0332-0d3d-44de-8fff-041036607ca6 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:01.267198 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef66253-8391-4bdb-95bf-71afa82dbfae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:01.268698 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef66253-8391-4bdb-95bf-71afa82dbfae admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:01.269039 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef66253-8391-4bdb-95bf-71afa82dbfae admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:01.269407 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef66253-8391-4bdb-95bf-71afa82dbfae admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:01.271117 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef66253-8391-4bdb-95bf-71afa82dbfae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:01.275263 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef66253-8391-4bdb-95bf-71afa82dbfae admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:01.297437 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1010/2011] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:47:01.299348 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69f092a-af67-407d-bd77-c63b9409bcac admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:01.316278 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69f092a-af67-407d-bd77-c63b9409bcac admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:01.322013 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e88b35b-fbe6-411d-889c-cfad81d3ef2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:01.323122 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88b35b-fbe6-411d-889c-cfad81d3ef2f admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:01.323538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88b35b-fbe6-411d-889c-cfad81d3ef2f admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:01.323922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88b35b-fbe6-411d-889c-cfad81d3ef2f admin admin] PATH_INFO: `/v3/projects/bd4f750fa0fe44dd91efff02a3ff0c37/users/2c6e96798c1343868a84ef0b16fb5a8c/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:01.341371 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e88b35b-fbe6-411d-889c-cfad81d3ef2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd4f750fa0fe44dd91efff02a3ff0c37, user_id=2c6e96798c1343868a84ef0b16fb5a8c, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:01.344335 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e88b35b-fbe6-411d-889c-cfad81d3ef2f admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:01.361815 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1002/2012] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:47:01 2022] PUT /identity/v3/projects/bd4f750fa0fe44dd91efff02a3ff0c37/users/2c6e96798c1343868a84ef0b16fb5a8c/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:01.372976 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e5a00-2a90-4cc3-a9df-027ef9f459d5 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:01.373303 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e5a00-2a90-4cc3-a9df-027ef9f459d5 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:01.373605 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e5a00-2a90-4cc3-a9df-027ef9f459d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:01.406513 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-ec0e5a00-2a90-4cc3-a9df-027ef9f459d5 None None] MFA Rules not processed for user `2c6e96798c1343868a84ef0b16fb5a8c`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:01.407891 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-ec0e5a00-2a90-4cc3-a9df-027ef9f459d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:01.490241 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:01.498666 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1011/2013] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:01.503380 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e88b35b-fbe6-411d-889c-cfad81d3ef2f admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:01.612876 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e88b35b-fbe6-411d-889c-cfad81d3ef2f admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:01.621522 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71da822f-ecf0-4c29-8af0-1e3fab95fe9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:01.622901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da822f-ecf0-4c29-8af0-1e3fab95fe9f admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:01.623344 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da822f-ecf0-4c29-8af0-1e3fab95fe9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:01.623797 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da822f-ecf0-4c29-8af0-1e3fab95fe9f admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:01.624856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71da822f-ecf0-4c29-8af0-1e3fab95fe9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:01.627378 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71da822f-ecf0-4c29-8af0-1e3fab95fe9f admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:01.668103 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1003/2014] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:47:01 2022] POST /identity/v3/projects => generated 387 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:47:01.669461 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e5a00-2a90-4cc3-a9df-027ef9f459d5 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:01.690858 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e5a00-2a90-4cc3-a9df-027ef9f459d5 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:01.782078 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c3d53-151b-4c5d-bab6-c7bd9c58c682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:01.782769 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c3d53-151b-4c5d-bab6-c7bd9c58c682 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:01.783425 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c3d53-151b-4c5d-bab6-c7bd9c58c682 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:01.783717 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c3d53-151b-4c5d-bab6-c7bd9c58c682 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:01.784452 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c3d53-151b-4c5d-bab6-c7bd9c58c682 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:01.787628 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c3d53-151b-4c5d-bab6-c7bd9c58c682 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:01.816776 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1012/2015] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:47:01 2022] POST /identity/v3/users => generated 300 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:01.821416 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71da822f-ecf0-4c29-8af0-1e3fab95fe9f admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:01.833104 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71da822f-ecf0-4c29-8af0-1e3fab95fe9f admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:01.838410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be08081-ebc1-42aa-9ba8-bca594895fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:01.839342 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be08081-ebc1-42aa-9ba8-bca594895fdd admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:01.839773 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be08081-ebc1-42aa-9ba8-bca594895fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:01.840185 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be08081-ebc1-42aa-9ba8-bca594895fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:01.841146 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be08081-ebc1-42aa-9ba8-bca594895fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:01.843042 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be08081-ebc1-42aa-9ba8-bca594895fdd admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:01.858144 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1004/2016] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:47:01.862877 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c3d53-151b-4c5d-bab6-c7bd9c58c682 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:01.875976 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c3d53-151b-4c5d-bab6-c7bd9c58c682 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:01.882265 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce59028-698c-41ef-9718-5ea713da707f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:01.883467 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce59028-698c-41ef-9718-5ea713da707f admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:01.884051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce59028-698c-41ef-9718-5ea713da707f admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:01.884474 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce59028-698c-41ef-9718-5ea713da707f admin admin] PATH_INFO: `/v3/projects/809c7e5b67204ee6b15d3bb2a160b0f7/users/c6abb359221f4d82a602e7913ae9300d/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:01.907413 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce59028-698c-41ef-9718-5ea713da707f admin admin] RBAC: Authorizing `identity:create_grant(project_id=809c7e5b67204ee6b15d3bb2a160b0f7, user_id=c6abb359221f4d82a602e7913ae9300d, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:01.911943 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce59028-698c-41ef-9718-5ea713da707f admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:01.933924 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1013/2017] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:47:01 2022] PUT /identity/v3/projects/809c7e5b67204ee6b15d3bb2a160b0f7/users/c6abb359221f4d82a602e7913ae9300d/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:01.935661 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be08081-ebc1-42aa-9ba8-bca594895fdd admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:01.948809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be08081-ebc1-42aa-9ba8-bca594895fdd admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:02.018954 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebcfa10-6788-4900-adc5-2ad20014a635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:02.019857 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebcfa10-6788-4900-adc5-2ad20014a635 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:02.021625 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebcfa10-6788-4900-adc5-2ad20014a635 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:02.022035 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebcfa10-6788-4900-adc5-2ad20014a635 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:02.023226 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebcfa10-6788-4900-adc5-2ad20014a635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:02.025197 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebcfa10-6788-4900-adc5-2ad20014a635 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:02.028422 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:02.043677 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce59028-698c-41ef-9718-5ea713da707f admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:02.046552 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1005/2018] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:47:02.059411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce59028-698c-41ef-9718-5ea713da707f admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:02.072832 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a5451-6a9f-49ee-9fe1-3110b6452a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:02.074247 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a5451-6a9f-49ee-9fe1-3110b6452a00 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:02.074604 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a5451-6a9f-49ee-9fe1-3110b6452a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:02.075035 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a5451-6a9f-49ee-9fe1-3110b6452a00 admin admin] PATH_INFO: `/v3/projects/809c7e5b67204ee6b15d3bb2a160b0f7/users/c6abb359221f4d82a602e7913ae9300d/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:02.079133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7a5451-6a9f-49ee-9fe1-3110b6452a00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=809c7e5b67204ee6b15d3bb2a160b0f7, user_id=c6abb359221f4d82a602e7913ae9300d, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:02.081982 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7a5451-6a9f-49ee-9fe1-3110b6452a00 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:02.105114 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1014/2019] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:47:02 2022] PUT /identity/v3/projects/809c7e5b67204ee6b15d3bb2a160b0f7/users/c6abb359221f4d82a602e7913ae9300d/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:02.106939 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a592fc-7431-4d1d-8861-bb079797c261 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:02.107235 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a592fc-7431-4d1d-8861-bb079797c261 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:02.107515 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a592fc-7431-4d1d-8861-bb079797c261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:02.137044 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-84a592fc-7431-4d1d-8861-bb079797c261 None None] MFA Rules not processed for user `c6abb359221f4d82a602e7913ae9300d`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:02.137778 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-84a592fc-7431-4d1d-8861-bb079797c261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:02.219953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1006/2020] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:02 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:02.232377 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a5451-6a9f-49ee-9fe1-3110b6452a00 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:02.234153 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-4e7a5451-6a9f-49ee-9fe1-3110b6452a00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:02.315408 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a5451-6a9f-49ee-9fe1-3110b6452a00 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:02.320395 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca0db9-914c-4929-bc45-6ec402c2a8ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:02.321089 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca0db9-914c-4929-bc45-6ec402c2a8ad service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:02.321379 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca0db9-914c-4929-bc45-6ec402c2a8ad service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:02.321659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca0db9-914c-4929-bc45-6ec402c2a8ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:02.331606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca0db9-914c-4929-bc45-6ec402c2a8ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:02.334760 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca0db9-914c-4929-bc45-6ec402c2a8ad service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:02.344533 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1015/2021] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:47:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:47:02.383452 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a592fc-7431-4d1d-8861-bb079797c261 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:02.468538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a592fc-7431-4d1d-8861-bb079797c261 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:02.475919 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41076848-a728-47a6-918b-1505de40e551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:02.477665 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41076848-a728-47a6-918b-1505de40e551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:02.478532 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41076848-a728-47a6-918b-1505de40e551 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:02.478532 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41076848-a728-47a6-918b-1505de40e551 admin admin] PATH_INFO: `/v3/users/2c6e96798c1343868a84ef0b16fb5a8c` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:02.479777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41076848-a728-47a6-918b-1505de40e551 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c6e96798c1343868a84ef0b16fb5a8c)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:02.482764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41076848-a728-47a6-918b-1505de40e551 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:02.491980 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca0db9-914c-4929-bc45-6ec402c2a8ad service cinder] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:02.493885 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-a2ca0db9-914c-4929-bc45-6ec402c2a8ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:02.510980 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca0db9-914c-4929-bc45-6ec402c2a8ad service cinder] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:02.522270 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b456e7a-ee05-43ed-9fd6-9e1106225bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:02.537678 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b456e7a-ee05-43ed-9fd6-9e1106225bd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:02.537962 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b456e7a-ee05-43ed-9fd6-9e1106225bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:02.538255 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b456e7a-ee05-43ed-9fd6-9e1106225bd2 admin admin] PATH_INFO: `/v3/projects/3717537d0c584ec18919b5039ce2c8ab` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:02.540501 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b456e7a-ee05-43ed-9fd6-9e1106225bd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3717537d0c584ec18919b5039ce2c8ab)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:02.542514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b456e7a-ee05-43ed-9fd6-9e1106225bd2 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:02.577872 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-41076848-a728-47a6-918b-1505de40e551 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c6e96798c1343868a84ef0b16fb5a8c'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:02.595424 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-41076848-a728-47a6-918b-1505de40e551 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c6e96798c1343868a84ef0b16fb5a8c'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:02.596302 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-41076848-a728-47a6-918b-1505de40e551 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c6e96798c1343868a84ef0b16fb5a8c'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:02.605564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:02.606958 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8b456e7a-ee05-43ed-9fd6-9e1106225bd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3717537d0c584ec18919b5039ce2c8ab', 'request_id': 'req-8b456e7a-ee05-43ed-9fd6-9e1106225bd2'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:02.608502 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-41076848-a728-47a6-918b-1505de40e551 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c6e96798c1343868a84ef0b16fb5a8c'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:02.613975 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8b456e7a-ee05-43ed-9fd6-9e1106225bd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3717537d0c584ec18919b5039ce2c8ab', 'request_id': 'req-8b456e7a-ee05-43ed-9fd6-9e1106225bd2'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:02.623692 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8b456e7a-ee05-43ed-9fd6-9e1106225bd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3717537d0c584ec18919b5039ce2c8ab', 'request_id': 'req-8b456e7a-ee05-43ed-9fd6-9e1106225bd2'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:02.628748 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-41076848-a728-47a6-918b-1505de40e551 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c6e96798c1343868a84ef0b16fb5a8c'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:02.637040 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1007/2022] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:47:02 2022] DELETE /identity/v3/users/2c6e96798c1343868a84ef0b16fb5a8c => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:02.641376 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8b456e7a-ee05-43ed-9fd6-9e1106225bd2 admin admin] The token cache is being invalidate because project 3717537d0c584ec18919b5039ce2c8ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:47:02.641745 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8b456e7a-ee05-43ed-9fd6-9e1106225bd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:02.642399 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe331f1-df09-497c-9b4b-c61e619a8548 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:02.642566 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8b456e7a-ee05-43ed-9fd6-9e1106225bd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:02.643490 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe331f1-df09-497c-9b4b-c61e619a8548 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:02.644051 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe331f1-df09-497c-9b4b-c61e619a8548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:02.649262 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1016/2023] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:47:02 2022] DELETE /identity/v3/projects/3717537d0c584ec18919b5039ce2c8ab => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:47:02.652175 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951028c7-dd72-4162-9944-93a66a21d2d9 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:02.652563 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951028c7-dd72-4162-9944-93a66a21d2d9 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:02.652965 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951028c7-dd72-4162-9944-93a66a21d2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:02.660217 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:02.678968 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-fbe331f1-df09-497c-9b4b-c61e619a8548 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:02.681196 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-fbe331f1-df09-497c-9b4b-c61e619a8548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:02.681512 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-951028c7-dd72-4162-9944-93a66a21d2d9 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:02.682881 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-951028c7-dd72-4162-9944-93a66a21d2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:02.765049 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1017/2024] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:02.773381 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8b882-bf65-4e5b-913e-04288499a78e None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:02.774562 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8b882-bf65-4e5b-913e-04288499a78e None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:02.774853 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8b882-bf65-4e5b-913e-04288499a78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:02.807233 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1008/2025] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:47:02.815355 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe331f1-df09-497c-9b4b-c61e619a8548 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:02.821188 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-ecb8b882-bf65-4e5b-913e-04288499a78e None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:02.822156 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-fbe331f1-df09-497c-9b4b-c61e619a8548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:02.822427 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-ecb8b882-bf65-4e5b-913e-04288499a78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:02.836957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1018/2026] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:02.868437 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8b882-bf65-4e5b-913e-04288499a78e None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:02.889041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8b882-bf65-4e5b-913e-04288499a78e None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:02.907627 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2dae43-61d6-4129-8de3-f7b4e8425f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:02.908805 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2dae43-61d6-4129-8de3-f7b4e8425f23 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:02.909265 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2dae43-61d6-4129-8de3-f7b4e8425f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:02.909666 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2dae43-61d6-4129-8de3-f7b4e8425f23 admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:02.910549 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2dae43-61d6-4129-8de3-f7b4e8425f23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:47:02.911618 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2dae43-61d6-4129-8de3-f7b4e8425f23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:02.915532 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2dae43-61d6-4129-8de3-f7b4e8425f23 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:02.933865 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1019/2027] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:47:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:47:02.935386 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2dae43-61d6-4129-8de3-f7b4e8425f23 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:02.948978 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe331f1-df09-497c-9b4b-c61e619a8548 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:02.954242 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae864df7-6e87-4cb8-8f32-9ce36bf96332 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:02.955296 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2dae43-61d6-4129-8de3-f7b4e8425f23 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:02.957148 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae864df7-6e87-4cb8-8f32-9ce36bf96332 service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:02.957647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae864df7-6e87-4cb8-8f32-9ce36bf96332 service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:02.960715 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae864df7-6e87-4cb8-8f32-9ce36bf96332 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:02.973230 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae864df7-6e87-4cb8-8f32-9ce36bf96332 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:02.983307 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae864df7-6e87-4cb8-8f32-9ce36bf96332 service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:02.989926 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a01c3-11fe-4426-a527-04bdfa380e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:02.990923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a01c3-11fe-4426-a527-04bdfa380e98 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:02.991334 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a01c3-11fe-4426-a527-04bdfa380e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:02.992448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a01c3-11fe-4426-a527-04bdfa380e98 admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:02.993330 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3a01c3-11fe-4426-a527-04bdfa380e98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:02.995991 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1009/2028] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:47:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:47:03.000975 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3a01c3-11fe-4426-a527-04bdfa380e98 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:03.050599 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1020/2029] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:47:02 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:03.055003 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae864df7-6e87-4cb8-8f32-9ce36bf96332 service neutron] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:03.065203 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae864df7-6e87-4cb8-8f32-9ce36bf96332 service neutron] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:03.165570 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7689818-8f26-42b7-8c3e-61815f428600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:03.166559 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7689818-8f26-42b7-8c3e-61815f428600 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:03.166998 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7689818-8f26-42b7-8c3e-61815f428600 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:03.167440 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7689818-8f26-42b7-8c3e-61815f428600 admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:03.168288 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7689818-8f26-42b7-8c3e-61815f428600 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:03.170444 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7689818-8f26-42b7-8c3e-61815f428600 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:03.202818 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1010/2030] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:47:03 2022] POST /identity/v3/users => generated 293 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:03.204101 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a01c3-11fe-4426-a527-04bdfa380e98 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:03.216507 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a01c3-11fe-4426-a527-04bdfa380e98 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:03.224164 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c970f2-51a5-4c81-824f-35568e59cec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:03.224912 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c970f2-51a5-4c81-824f-35568e59cec4 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:03.225491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c970f2-51a5-4c81-824f-35568e59cec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:03.225913 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c970f2-51a5-4c81-824f-35568e59cec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:03.227596 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c970f2-51a5-4c81-824f-35568e59cec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:03.229511 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c970f2-51a5-4c81-824f-35568e59cec4 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:03.249942 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1021/2031] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:03 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:47:03.252136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7689818-8f26-42b7-8c3e-61815f428600 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:03.264515 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7689818-8f26-42b7-8c3e-61815f428600 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:03.272363 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1583a4b5-9874-4851-b32a-17e017ef198c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:03.273480 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583a4b5-9874-4851-b32a-17e017ef198c admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:03.273911 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583a4b5-9874-4851-b32a-17e017ef198c admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:03.274450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583a4b5-9874-4851-b32a-17e017ef198c admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:03.275509 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1583a4b5-9874-4851-b32a-17e017ef198c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:03.277704 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1583a4b5-9874-4851-b32a-17e017ef198c admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:03.296642 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c970f2-51a5-4c81-824f-35568e59cec4 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:03.306919 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1011/2032] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:03 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:47:03.311224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c970f2-51a5-4c81-824f-35568e59cec4 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:03.316942 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7ee32-2b77-481a-bb49-a8caebb4909d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:03.319000 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7ee32-2b77-481a-bb49-a8caebb4909d admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:03.319447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7ee32-2b77-481a-bb49-a8caebb4909d admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:03.319930 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7ee32-2b77-481a-bb49-a8caebb4909d admin admin] PATH_INFO: `/v3/projects/fc641a408c23442e8d5d89c672e00c7a/users/2a47cdf4a7e541eabb1d5019a2cff46a/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:03.355044 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f7ee32-2b77-481a-bb49-a8caebb4909d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc641a408c23442e8d5d89c672e00c7a, user_id=2a47cdf4a7e541eabb1d5019a2cff46a, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:03.358291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f7ee32-2b77-481a-bb49-a8caebb4909d admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:03.394364 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1022/2033] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:47:03 2022] PUT /identity/v3/projects/fc641a408c23442e8d5d89c672e00c7a/users/2a47cdf4a7e541eabb1d5019a2cff46a/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:03.404985 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32dfad-5466-4a07-8c42-2f7c83ae84d1 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:03.405382 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32dfad-5466-4a07-8c42-2f7c83ae84d1 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:03.405734 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32dfad-5466-4a07-8c42-2f7c83ae84d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:03.462233 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-1a32dfad-5466-4a07-8c42-2f7c83ae84d1 None None] MFA Rules not processed for user `2a47cdf4a7e541eabb1d5019a2cff46a`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:03.463575 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-1a32dfad-5466-4a07-8c42-2f7c83ae84d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:03.481450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:03.614983 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1012/2034] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:03.616475 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7ee32-2b77-481a-bb49-a8caebb4909d admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:03.735718 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7ee32-2b77-481a-bb49-a8caebb4909d admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:03.766734 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c108cfb-7f01-4906-96de-34e5574a221f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:03.768280 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c108cfb-7f01-4906-96de-34e5574a221f admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:03.768566 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c108cfb-7f01-4906-96de-34e5574a221f admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:03.769417 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c108cfb-7f01-4906-96de-34e5574a221f admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:03.770144 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c108cfb-7f01-4906-96de-34e5574a221f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:03.772330 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c108cfb-7f01-4906-96de-34e5574a221f admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:03.777597 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:03.826167 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1023/2035] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:47:03 2022] POST /identity/v3/projects => generated 387 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:03.827635 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32dfad-5466-4a07-8c42-2f7c83ae84d1 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:03.847501 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32dfad-5466-4a07-8c42-2f7c83ae84d1 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:03.967306 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8c560e-7af2-473e-adef-5076ef77e821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:03.968578 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c560e-7af2-473e-adef-5076ef77e821 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:03.969238 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c560e-7af2-473e-adef-5076ef77e821 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:03.969730 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c560e-7af2-473e-adef-5076ef77e821 admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:03.970922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8c560e-7af2-473e-adef-5076ef77e821 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:03.973195 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8c560e-7af2-473e-adef-5076ef77e821 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:04.001206 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1013/2036] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:47:03 2022] POST /identity/v3/users => generated 294 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:04.005157 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c108cfb-7f01-4906-96de-34e5574a221f admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:04.019994 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c108cfb-7f01-4906-96de-34e5574a221f admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:04.031777 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33366dc2-d3c7-443a-83df-d144cfd3f218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:04.032478 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33366dc2-d3c7-443a-83df-d144cfd3f218 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:04.032766 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33366dc2-d3c7-443a-83df-d144cfd3f218 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:04.033070 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33366dc2-d3c7-443a-83df-d144cfd3f218 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:04.033923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33366dc2-d3c7-443a-83df-d144cfd3f218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:04.035647 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33366dc2-d3c7-443a-83df-d144cfd3f218 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:04.065457 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1024/2037] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:04 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:47:04.067690 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8c560e-7af2-473e-adef-5076ef77e821 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:04.084346 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8c560e-7af2-473e-adef-5076ef77e821 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:04.097673 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548cd356-d071-4397-9e3c-96edb4bf4a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:04.099153 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cd356-d071-4397-9e3c-96edb4bf4a3f admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:04.099569 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cd356-d071-4397-9e3c-96edb4bf4a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:04.099951 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cd356-d071-4397-9e3c-96edb4bf4a3f admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:04.101065 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548cd356-d071-4397-9e3c-96edb4bf4a3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:04.104835 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548cd356-d071-4397-9e3c-96edb4bf4a3f admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:04.134855 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1014/2038] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:04 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:47:04.137388 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33366dc2-d3c7-443a-83df-d144cfd3f218 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:04.158107 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33366dc2-d3c7-443a-83df-d144cfd3f218 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:04.171671 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e559e4d-538e-4611-986c-d7eb58272941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:04.173044 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e559e4d-538e-4611-986c-d7eb58272941 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:04.173494 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e559e4d-538e-4611-986c-d7eb58272941 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:04.173917 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e559e4d-538e-4611-986c-d7eb58272941 admin admin] PATH_INFO: `/v3/projects/3a1c0eb8675f468a9a06f4ea8969437c/users/20be1661539244d389426433cd192e65/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:04.202057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548cd356-d071-4397-9e3c-96edb4bf4a3f admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:04.204283 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-548cd356-d071-4397-9e3c-96edb4bf4a3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:04.205510 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e559e4d-538e-4611-986c-d7eb58272941 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a1c0eb8675f468a9a06f4ea8969437c, user_id=20be1661539244d389426433cd192e65, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:04.208050 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e559e4d-538e-4611-986c-d7eb58272941 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:04.225975 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1025/2039] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:47:04 2022] PUT /identity/v3/projects/3a1c0eb8675f468a9a06f4ea8969437c/users/20be1661539244d389426433cd192e65/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:04.228242 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548cd356-d071-4397-9e3c-96edb4bf4a3f admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:04.237634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5e70b-5f5b-4ace-b460-540765424eae None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:04.237931 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5e70b-5f5b-4ace-b460-540765424eae None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:04.238213 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5e70b-5f5b-4ace-b460-540765424eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:04.285232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-10a5e70b-5f5b-4ace-b460-540765424eae None None] MFA Rules not processed for user `20be1661539244d389426433cd192e65`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:04.287393 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-10a5e70b-5f5b-4ace-b460-540765424eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:04.354171 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853bce15-4f88-4e84-abd7-118d4ee31972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:04.355202 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bce15-4f88-4e84-abd7-118d4ee31972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:04.355594 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bce15-4f88-4e84-abd7-118d4ee31972 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:04.356011 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bce15-4f88-4e84-abd7-118d4ee31972 admin admin] PATH_INFO: `/v3/projects/bd4f750fa0fe44dd91efff02a3ff0c37` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:04.363195 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853bce15-4f88-4e84-abd7-118d4ee31972 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd4f750fa0fe44dd91efff02a3ff0c37)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:04.365477 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853bce15-4f88-4e84-abd7-118d4ee31972 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:04.453917 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1026/2040] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:47:04.458942 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:04.459585 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5e70b-5f5b-4ace-b460-540765424eae None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:04.477979 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-853bce15-4f88-4e84-abd7-118d4ee31972 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd4f750fa0fe44dd91efff02a3ff0c37', 'request_id': 'req-853bce15-4f88-4e84-abd7-118d4ee31972'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:04.485291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5e70b-5f5b-4ace-b460-540765424eae None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:04.485767 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-853bce15-4f88-4e84-abd7-118d4ee31972 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd4f750fa0fe44dd91efff02a3ff0c37', 'request_id': 'req-853bce15-4f88-4e84-abd7-118d4ee31972'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:04.498165 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-853bce15-4f88-4e84-abd7-118d4ee31972 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd4f750fa0fe44dd91efff02a3ff0c37', 'request_id': 'req-853bce15-4f88-4e84-abd7-118d4ee31972'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:04.513483 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-853bce15-4f88-4e84-abd7-118d4ee31972 admin admin] The token cache is being invalidate because project bd4f750fa0fe44dd91efff02a3ff0c37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:47:04.514457 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-853bce15-4f88-4e84-abd7-118d4ee31972 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:04.515191 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-853bce15-4f88-4e84-abd7-118d4ee31972 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:04.521358 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1015/2041] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:47:04 2022] DELETE /identity/v3/projects/bd4f750fa0fe44dd91efff02a3ff0c37 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:47:04.522116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853bce15-4f88-4e84-abd7-118d4ee31972 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:04.525050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-853bce15-4f88-4e84-abd7-118d4ee31972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:04.583149 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34698cf1-3c2c-4433-a159-f319e7be9286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:04.584342 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34698cf1-3c2c-4433-a159-f319e7be9286 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:04.584629 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34698cf1-3c2c-4433-a159-f319e7be9286 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:04.584908 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34698cf1-3c2c-4433-a159-f319e7be9286 admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:04.588768 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34698cf1-3c2c-4433-a159-f319e7be9286 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:04.591123 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34698cf1-3c2c-4433-a159-f319e7be9286 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:04.624541 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1027/2042] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:47:04 2022] POST /identity/v3/projects => generated 387 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:04.627111 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853bce15-4f88-4e84-abd7-118d4ee31972 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:04.628933 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34698cf1-3c2c-4433-a159-f319e7be9286 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:04.631724 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-34698cf1-3c2c-4433-a159-f319e7be9286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:04.678481 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:04.722810 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34698cf1-3c2c-4433-a159-f319e7be9286 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:04.729360 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8297b375-8bba-4e2d-9e59-d6d48e12f48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:04.730632 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2975a77-0b52-4bf3-9df9-5792493d918c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:04.730954 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297b375-8bba-4e2d-9e59-d6d48e12f48b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:04.731405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2975a77-0b52-4bf3-9df9-5792493d918c admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:04.731593 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297b375-8bba-4e2d-9e59-d6d48e12f48b admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:04.731701 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2975a77-0b52-4bf3-9df9-5792493d918c admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:04.731953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2975a77-0b52-4bf3-9df9-5792493d918c admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:04.732246 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297b375-8bba-4e2d-9e59-d6d48e12f48b admin admin] PATH_INFO: `/v3/users/c6abb359221f4d82a602e7913ae9300d` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:04.732663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2975a77-0b52-4bf3-9df9-5792493d918c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:04.733819 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8297b375-8bba-4e2d-9e59-d6d48e12f48b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6abb359221f4d82a602e7913ae9300d)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:04.735383 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2975a77-0b52-4bf3-9df9-5792493d918c admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:04.736793 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8297b375-8bba-4e2d-9e59-d6d48e12f48b admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:04.765860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1028/2043] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:47:04 2022] POST /identity/v3/users => generated 300 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:04.769773 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2975a77-0b52-4bf3-9df9-5792493d918c admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:04.785501 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2975a77-0b52-4bf3-9df9-5792493d918c admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:04.794777 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a447a34-ffc5-4cda-9d81-2ba90f6323f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:04.796536 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a447a34-ffc5-4cda-9d81-2ba90f6323f7 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:04.796927 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a447a34-ffc5-4cda-9d81-2ba90f6323f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:04.797384 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a447a34-ffc5-4cda-9d81-2ba90f6323f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:04.798432 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a447a34-ffc5-4cda-9d81-2ba90f6323f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:04.800187 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a447a34-ffc5-4cda-9d81-2ba90f6323f7 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:04.817209 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1029/2044] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:04 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:47:04.821692 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a447a34-ffc5-4cda-9d81-2ba90f6323f7 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:04.837750 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a447a34-ffc5-4cda-9d81-2ba90f6323f7 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:04.844226 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1371ac-a064-4062-8566-3eba1f5861fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:04.845448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1371ac-a064-4062-8566-3eba1f5861fe admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:04.845841 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1371ac-a064-4062-8566-3eba1f5861fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:04.846226 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1371ac-a064-4062-8566-3eba1f5861fe admin admin] PATH_INFO: `/v3/projects/65cfa71f869e42f6b3430149f7264129/users/2c91cc24faa44fd68ddb05222106b67b/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:04.871453 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-8297b375-8bba-4e2d-9e59-d6d48e12f48b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6abb359221f4d82a602e7913ae9300d'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:04.872564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1371ac-a064-4062-8566-3eba1f5861fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=65cfa71f869e42f6b3430149f7264129, user_id=2c91cc24faa44fd68ddb05222106b67b, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:04.881575 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1371ac-a064-4062-8566-3eba1f5861fe admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:04.884085 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-8297b375-8bba-4e2d-9e59-d6d48e12f48b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6abb359221f4d82a602e7913ae9300d'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:04.884865 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-8297b375-8bba-4e2d-9e59-d6d48e12f48b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6abb359221f4d82a602e7913ae9300d'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:04.902737 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1030/2045] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:47:04 2022] PUT /identity/v3/projects/65cfa71f869e42f6b3430149f7264129/users/2c91cc24faa44fd68ddb05222106b67b/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:47:04.906166 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1371ac-a064-4062-8566-3eba1f5861fe admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:04.908515 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-bc1371ac-a064-4062-8566-3eba1f5861fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:04.919853 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-8297b375-8bba-4e2d-9e59-d6d48e12f48b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6abb359221f4d82a602e7913ae9300d'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:04.932368 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:04.944844 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-8297b375-8bba-4e2d-9e59-d6d48e12f48b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6abb359221f4d82a602e7913ae9300d'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:04.952365 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1016/2046] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:47:04 2022] DELETE /identity/v3/users/c6abb359221f4d82a602e7913ae9300d => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:05.027695 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1371ac-a064-4062-8566-3eba1f5861fe admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:05.146717 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b61b177-caa9-4b62-b576-78d420320235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:05.147141 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61b177-caa9-4b62-b576-78d420320235 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:05.147360 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61b177-caa9-4b62-b576-78d420320235 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:05.147645 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61b177-caa9-4b62-b576-78d420320235 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:05.148786 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b61b177-caa9-4b62-b576-78d420320235 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:05.159096 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b61b177-caa9-4b62-b576-78d420320235 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:05.184654 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1031/2047] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:04 2022] GET /identity/v3/roles => generated 1420 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:47:05.186651 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8297b375-8bba-4e2d-9e59-d6d48e12f48b admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:05.200081 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8297b375-8bba-4e2d-9e59-d6d48e12f48b admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:05.206437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5101455-fe58-48f3-acd2-5238f42188f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:05.207466 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5101455-fe58-48f3-acd2-5238f42188f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:05.208415 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5101455-fe58-48f3-acd2-5238f42188f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:05.208874 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5101455-fe58-48f3-acd2-5238f42188f5 admin admin] PATH_INFO: `/v3/projects/65cfa71f869e42f6b3430149f7264129/users/2c91cc24faa44fd68ddb05222106b67b/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:05.211620 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5101455-fe58-48f3-acd2-5238f42188f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65cfa71f869e42f6b3430149f7264129, user_id=2c91cc24faa44fd68ddb05222106b67b, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:05.215555 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5101455-fe58-48f3-acd2-5238f42188f5 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:05.233523 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1017/2048] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:47:05 2022] PUT /identity/v3/projects/65cfa71f869e42f6b3430149f7264129/users/2c91cc24faa44fd68ddb05222106b67b/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:05.246407 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f90ce-bf23-4a91-b06a-840a2d5fc2bf None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:05.246734 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f90ce-bf23-4a91-b06a-840a2d5fc2bf None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:05.247031 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f90ce-bf23-4a91-b06a-840a2d5fc2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:05.293631 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-ce7f90ce-bf23-4a91-b06a-840a2d5fc2bf None None] MFA Rules not processed for user `2c91cc24faa44fd68ddb05222106b67b`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:05.293888 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-ce7f90ce-bf23-4a91-b06a-840a2d5fc2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:05.309849 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:05.389748 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1032/2049] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:05 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:05.400460 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5101455-fe58-48f3-acd2-5238f42188f5 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:05.402763 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a5101455-fe58-48f3-acd2-5238f42188f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:05.424956 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f90ce-bf23-4a91-b06a-840a2d5fc2bf None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:05.428526 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-ce7f90ce-bf23-4a91-b06a-840a2d5fc2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:05.567180 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5101455-fe58-48f3-acd2-5238f42188f5 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:05.572511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b7345-5250-43a6-871d-3c01e1436070 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:05.572847 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f90ce-bf23-4a91-b06a-840a2d5fc2bf None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:05.573677 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b7345-5250-43a6-871d-3c01e1436070 service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:05.574217 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b7345-5250-43a6-871d-3c01e1436070 service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:05.574910 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b7345-5250-43a6-871d-3c01e1436070 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:05.581497 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e2b44-dbe1-4012-8709-ad550fe04317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:05.583441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e2b44-dbe1-4012-8709-ad550fe04317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:05.584396 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e2b44-dbe1-4012-8709-ad550fe04317 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:05.585269 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e2b44-dbe1-4012-8709-ad550fe04317 admin admin] PATH_INFO: `/v3/projects/809c7e5b67204ee6b15d3bb2a160b0f7` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:05.586794 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e2b44-dbe1-4012-8709-ad550fe04317 admin admin] RBAC: Authorizing `identity:delete_project(project_id=809c7e5b67204ee6b15d3bb2a160b0f7)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:05.592429 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e2b44-dbe1-4012-8709-ad550fe04317 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:05.594785 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b7345-5250-43a6-871d-3c01e1436070 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:05.597156 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b7345-5250-43a6-871d-3c01e1436070 service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:05.613716 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1018/2050] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:47:05 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:47:05.672673 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-632e2b44-dbe1-4012-8709-ad550fe04317 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '809c7e5b67204ee6b15d3bb2a160b0f7', 'request_id': 'req-632e2b44-dbe1-4012-8709-ad550fe04317'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:05.683105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-632e2b44-dbe1-4012-8709-ad550fe04317 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '809c7e5b67204ee6b15d3bb2a160b0f7', 'request_id': 'req-632e2b44-dbe1-4012-8709-ad550fe04317'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:05.709949 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-632e2b44-dbe1-4012-8709-ad550fe04317 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '809c7e5b67204ee6b15d3bb2a160b0f7', 'request_id': 'req-632e2b44-dbe1-4012-8709-ad550fe04317'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:05.728579 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-632e2b44-dbe1-4012-8709-ad550fe04317 admin admin] The token cache is being invalidate because project 809c7e5b67204ee6b15d3bb2a160b0f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:47:05.728879 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-632e2b44-dbe1-4012-8709-ad550fe04317 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:05.735564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-632e2b44-dbe1-4012-8709-ad550fe04317 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:05.737544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1033/2051] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:47:05 2022] DELETE /identity/v3/projects/809c7e5b67204ee6b15d3bb2a160b0f7 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:05.743083 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce83dd-4be0-401e-ae9c-42cf2beb49a6 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:05.743444 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce83dd-4be0-401e-ae9c-42cf2beb49a6 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:05.743820 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce83dd-4be0-401e-ae9c-42cf2beb49a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:05.763378 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-5bce83dd-4be0-401e-ae9c-42cf2beb49a6 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:05.764983 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-5bce83dd-4be0-401e-ae9c-42cf2beb49a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:05.853672 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1019/2052] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:05.859286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ab0ab-203f-4d25-9163-05b07b7f88e1 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:05.859766 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ab0ab-203f-4d25-9163-05b07b7f88e1 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:05.860131 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ab0ab-203f-4d25-9163-05b07b7f88e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:05.884448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-425ab0ab-203f-4d25-9163-05b07b7f88e1 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:05.885715 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-425ab0ab-203f-4d25-9163-05b07b7f88e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:05.895038 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1034/2053] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:47:05.899771 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce83dd-4be0-401e-ae9c-42cf2beb49a6 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:05.916731 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce83dd-4be0-401e-ae9c-42cf2beb49a6 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:05.922373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2295a15-7a48-4a2c-bb74-cc564e8bd58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:05.923069 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2295a15-7a48-4a2c-bb74-cc564e8bd58b admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:05.923477 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2295a15-7a48-4a2c-bb74-cc564e8bd58b admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:05.923878 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2295a15-7a48-4a2c-bb74-cc564e8bd58b admin admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:05.924733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2295a15-7a48-4a2c-bb74-cc564e8bd58b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:47:05.925220 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2295a15-7a48-4a2c-bb74-cc564e8bd58b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:05.927503 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2295a15-7a48-4a2c-bb74-cc564e8bd58b admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:05.942980 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1020/2054] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:47:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:47:05.947712 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ab0ab-203f-4d25-9163-05b07b7f88e1 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:05.956970 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ab0ab-203f-4d25-9163-05b07b7f88e1 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:05.962494 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95560f10-44e2-4495-802f-ed815cfcc708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:05.963223 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95560f10-44e2-4495-802f-ed815cfcc708 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:05.963604 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95560f10-44e2-4495-802f-ed815cfcc708 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:05.963949 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95560f10-44e2-4495-802f-ed815cfcc708 admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:05.964728 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95560f10-44e2-4495-802f-ed815cfcc708 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:05.967060 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95560f10-44e2-4495-802f-ed815cfcc708 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:06.000145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1035/2055] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:47:05 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:47:06.001686 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2295a15-7a48-4a2c-bb74-cc564e8bd58b admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:06.012977 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2295a15-7a48-4a2c-bb74-cc564e8bd58b admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:06.078552 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715850ea-56bc-44d4-9a0a-df3c009dfe2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:06.078728 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715850ea-56bc-44d4-9a0a-df3c009dfe2f admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:06.078999 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715850ea-56bc-44d4-9a0a-df3c009dfe2f admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:06.079549 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715850ea-56bc-44d4-9a0a-df3c009dfe2f admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:06.080284 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715850ea-56bc-44d4-9a0a-df3c009dfe2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:06.082213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715850ea-56bc-44d4-9a0a-df3c009dfe2f admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:06.093778 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:06.121167 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1021/2056] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:47:06 2022] POST /identity/v3/users => generated 290 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:06.122893 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95560f10-44e2-4495-802f-ed815cfcc708 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:06.134400 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95560f10-44e2-4495-802f-ed815cfcc708 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:06.144425 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de9f92-751d-451c-bcd6-415045d0bb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:06.145947 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de9f92-751d-451c-bcd6-415045d0bb01 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:06.146254 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de9f92-751d-451c-bcd6-415045d0bb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:06.146660 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de9f92-751d-451c-bcd6-415045d0bb01 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:06.147405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de9f92-751d-451c-bcd6-415045d0bb01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:06.149088 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de9f92-751d-451c-bcd6-415045d0bb01 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:06.151514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:06.164611 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1036/2057] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:47:06.165813 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715850ea-56bc-44d4-9a0a-df3c009dfe2f admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:06.176447 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715850ea-56bc-44d4-9a0a-df3c009dfe2f admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:06.181931 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19ce3c-5111-4d07-b1e4-2195b1606cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:06.182627 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19ce3c-5111-4d07-b1e4-2195b1606cc2 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:06.182910 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19ce3c-5111-4d07-b1e4-2195b1606cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:06.183184 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19ce3c-5111-4d07-b1e4-2195b1606cc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:06.183907 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee19ce3c-5111-4d07-b1e4-2195b1606cc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:06.185525 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee19ce3c-5111-4d07-b1e4-2195b1606cc2 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:06.197047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1022/2058] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:47:06.201203 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de9f92-751d-451c-bcd6-415045d0bb01 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:06.212168 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de9f92-751d-451c-bcd6-415045d0bb01 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:06.224594 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a7168-170e-4708-8589-c5793d9b0035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:06.225524 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a7168-170e-4708-8589-c5793d9b0035 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:06.225807 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a7168-170e-4708-8589-c5793d9b0035 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:06.226080 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a7168-170e-4708-8589-c5793d9b0035 admin admin] PATH_INFO: `/v3/projects/a4acbd65fe8b458fb0d72843a3c8483c/users/83b878ea1cce4774b3f60654e0fd672b/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:06.226388 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19ce3c-5111-4d07-b1e4-2195b1606cc2 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:06.228239 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-ee19ce3c-5111-4d07-b1e4-2195b1606cc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:06.241392 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a7168-170e-4708-8589-c5793d9b0035 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4acbd65fe8b458fb0d72843a3c8483c, user_id=83b878ea1cce4774b3f60654e0fd672b, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:06.244042 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a7168-170e-4708-8589-c5793d9b0035 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:06.259723 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1037/2059] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:47:06 2022] PUT /identity/v3/projects/a4acbd65fe8b458fb0d72843a3c8483c/users/83b878ea1cce4774b3f60654e0fd672b/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:47:06.271539 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb230f2-0a77-4703-bbc4-185c4fd120e8 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:06.271830 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb230f2-0a77-4703-bbc4-185c4fd120e8 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:06.272105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb230f2-0a77-4703-bbc4-185c4fd120e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:06.304282 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-3fb230f2-0a77-4703-bbc4-185c4fd120e8 None None] MFA Rules not processed for user `83b878ea1cce4774b3f60654e0fd672b`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:06.304282 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-3fb230f2-0a77-4703-bbc4-185c4fd120e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:06.316119 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19ce3c-5111-4d07-b1e4-2195b1606cc2 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:06.391047 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1038/2060] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:47:06.392328 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e3751-cc05-4b1b-8c7f-3b73b3bab90c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:06.392803 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb230f2-0a77-4703-bbc4-185c4fd120e8 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:06.393350 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e3751-cc05-4b1b-8c7f-3b73b3bab90c service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:06.393779 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e3751-cc05-4b1b-8c7f-3b73b3bab90c service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:06.394197 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e3751-cc05-4b1b-8c7f-3b73b3bab90c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:06.396375 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-143e3751-cc05-4b1b-8c7f-3b73b3bab90c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:06.488442 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143e3751-cc05-4b1b-8c7f-3b73b3bab90c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:06.490633 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143e3751-cc05-4b1b-8c7f-3b73b3bab90c service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:06.495580 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb230f2-0a77-4703-bbc4-185c4fd120e8 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:06.500688 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1023/2061] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:47:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:47:06.503933 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45050b-794b-4145-9c89-c959989a70ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:06.504723 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45050b-794b-4145-9c89-c959989a70ea admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:06.505223 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45050b-794b-4145-9c89-c959989a70ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:06.505570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45050b-794b-4145-9c89-c959989a70ea admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:06.506341 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45050b-794b-4145-9c89-c959989a70ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:06.508607 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45050b-794b-4145-9c89-c959989a70ea admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:06.535606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1039/2062] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:47:06 2022] POST /identity/v3/projects => generated 381 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:06.539772 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e3751-cc05-4b1b-8c7f-3b73b3bab90c service cinder] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:06.554587 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e3751-cc05-4b1b-8c7f-3b73b3bab90c service cinder] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:06.632063 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:06.646703 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c7f4c-d1e8-4f58-9f40-d8f82979f94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:06.649051 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c7f4c-d1e8-4f58-9f40-d8f82979f94c admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:06.649337 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c7f4c-d1e8-4f58-9f40-d8f82979f94c admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:06.649718 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c7f4c-d1e8-4f58-9f40-d8f82979f94c admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:06.650563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708c7f4c-d1e8-4f58-9f40-d8f82979f94c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:06.653110 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708c7f4c-d1e8-4f58-9f40-d8f82979f94c admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:06.678996 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1024/2063] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:47:06 2022] POST /identity/v3/users => generated 297 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:47:06.683288 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45050b-794b-4145-9c89-c959989a70ea admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:06.701078 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45050b-794b-4145-9c89-c959989a70ea admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:06.707296 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd3402-1f0a-498b-9307-ed40fb6c074f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:06.708501 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd3402-1f0a-498b-9307-ed40fb6c074f admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:06.708811 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd3402-1f0a-498b-9307-ed40fb6c074f admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:06.709158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd3402-1f0a-498b-9307-ed40fb6c074f admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:06.709897 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd3402-1f0a-498b-9307-ed40fb6c074f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:06.711573 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd3402-1f0a-498b-9307-ed40fb6c074f admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:06.737536 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1040/2064] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:06 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:47:06.744306 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c7f4c-d1e8-4f58-9f40-d8f82979f94c admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:06.758066 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c7f4c-d1e8-4f58-9f40-d8f82979f94c admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:06.763355 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99456418-a917-434a-954c-4c1b641e6247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:06.764479 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99456418-a917-434a-954c-4c1b641e6247 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:06.764859 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99456418-a917-434a-954c-4c1b641e6247 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:06.765276 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99456418-a917-434a-954c-4c1b641e6247 admin admin] PATH_INFO: `/v3/projects/3661728af2cd45ea81fd134b63f17a67/users/b37120b3f2ff489095fabeb7f7526c34/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:06.790532 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99456418-a917-434a-954c-4c1b641e6247 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3661728af2cd45ea81fd134b63f17a67, user_id=b37120b3f2ff489095fabeb7f7526c34, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:06.795752 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99456418-a917-434a-954c-4c1b641e6247 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:06.816133 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1025/2065] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:47:06 2022] PUT /identity/v3/projects/3661728af2cd45ea81fd134b63f17a67/users/b37120b3f2ff489095fabeb7f7526c34/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:06.820666 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd3402-1f0a-498b-9307-ed40fb6c074f admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:06.834484 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd3402-1f0a-498b-9307-ed40fb6c074f admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:06.918443 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:06.951997 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29da0ce-fb63-440e-9ae9-a5ac6d920e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:06.952715 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29da0ce-fb63-440e-9ae9-a5ac6d920e47 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:06.953102 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29da0ce-fb63-440e-9ae9-a5ac6d920e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:06.953456 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29da0ce-fb63-440e-9ae9-a5ac6d920e47 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:06.954366 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29da0ce-fb63-440e-9ae9-a5ac6d920e47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:06.959342 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29da0ce-fb63-440e-9ae9-a5ac6d920e47 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:06.973788 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1041/2066] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:06 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:47:06.978170 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99456418-a917-434a-954c-4c1b641e6247 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:06.989940 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99456418-a917-434a-954c-4c1b641e6247 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:06.994748 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4511a97-7a73-4af7-841f-7d88c5227893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:06.995685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4511a97-7a73-4af7-841f-7d88c5227893 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:06.995963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4511a97-7a73-4af7-841f-7d88c5227893 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:06.996236 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4511a97-7a73-4af7-841f-7d88c5227893 admin admin] PATH_INFO: `/v3/projects/3661728af2cd45ea81fd134b63f17a67/users/b37120b3f2ff489095fabeb7f7526c34/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:06.998259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4511a97-7a73-4af7-841f-7d88c5227893 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3661728af2cd45ea81fd134b63f17a67, user_id=b37120b3f2ff489095fabeb7f7526c34, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:07.001132 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4511a97-7a73-4af7-841f-7d88c5227893 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:07.012935 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1026/2067] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:47:06 2022] PUT /identity/v3/projects/3661728af2cd45ea81fd134b63f17a67/users/b37120b3f2ff489095fabeb7f7526c34/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:07.025075 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e641445-9b12-4760-a60f-42d559cb8ba5 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:07.025616 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e641445-9b12-4760-a60f-42d559cb8ba5 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:07.025961 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e641445-9b12-4760-a60f-42d559cb8ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:07.057139 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-1e641445-9b12-4760-a60f-42d559cb8ba5 None None] MFA Rules not processed for user `b37120b3f2ff489095fabeb7f7526c34`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:07.058450 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-1e641445-9b12-4760-a60f-42d559cb8ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:07.139996 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1042/2068] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:07.151157 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4511a97-7a73-4af7-841f-7d88c5227893 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:07.154018 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-b4511a97-7a73-4af7-841f-7d88c5227893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:07.219745 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4511a97-7a73-4af7-841f-7d88c5227893 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:07.224269 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e3f3c-7f4d-42bf-8433-7b5b3dffde50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:07.225025 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e3f3c-7f4d-42bf-8433-7b5b3dffde50 service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:07.225361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e3f3c-7f4d-42bf-8433-7b5b3dffde50 service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:07.225652 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e3f3c-7f4d-42bf-8433-7b5b3dffde50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:07.235039 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e3f3c-7f4d-42bf-8433-7b5b3dffde50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:07.236542 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e3f3c-7f4d-42bf-8433-7b5b3dffde50 service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:07.246915 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1027/2069] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:47:07 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:47:07.688591 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e641445-9b12-4760-a60f-42d559cb8ba5 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:07.752599 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e641445-9b12-4760-a60f-42d559cb8ba5 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:07.757970 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f36ac-d488-4dad-ae38-27dc0df7e285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:07.758837 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f36ac-d488-4dad-ae38-27dc0df7e285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:07.759145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f36ac-d488-4dad-ae38-27dc0df7e285 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:07.759511 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f36ac-d488-4dad-ae38-27dc0df7e285 admin admin] PATH_INFO: `/v3/users/83b878ea1cce4774b3f60654e0fd672b` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:07.760604 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f36ac-d488-4dad-ae38-27dc0df7e285 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83b878ea1cce4774b3f60654e0fd672b)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:07.762548 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f36ac-d488-4dad-ae38-27dc0df7e285 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:07.815751 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-d96f36ac-d488-4dad-ae38-27dc0df7e285 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83b878ea1cce4774b3f60654e0fd672b'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:07.825364 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-d96f36ac-d488-4dad-ae38-27dc0df7e285 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83b878ea1cce4774b3f60654e0fd672b'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:07.833774 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:07.839184 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-d96f36ac-d488-4dad-ae38-27dc0df7e285 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83b878ea1cce4774b3f60654e0fd672b'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:07.840621 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-d96f36ac-d488-4dad-ae38-27dc0df7e285 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83b878ea1cce4774b3f60654e0fd672b'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:07.860366 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-d96f36ac-d488-4dad-ae38-27dc0df7e285 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83b878ea1cce4774b3f60654e0fd672b'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:07.867451 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1043/2070] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:47:07 2022] DELETE /identity/v3/users/83b878ea1cce4774b3f60654e0fd672b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:07.872190 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562f9c8-ce4d-4f47-bfbc-b010279feb49 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:07.872654 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562f9c8-ce4d-4f47-bfbc-b010279feb49 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:07.872767 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562f9c8-ce4d-4f47-bfbc-b010279feb49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:07.890553 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-a562f9c8-ce4d-4f47-bfbc-b010279feb49 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:07.891908 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a562f9c8-ce4d-4f47-bfbc-b010279feb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:07.967095 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1028/2071] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:07.975700 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f36ac-d488-4dad-ae38-27dc0df7e285 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:07.977280 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-d96f36ac-d488-4dad-ae38-27dc0df7e285 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:08.050627 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f36ac-d488-4dad-ae38-27dc0df7e285 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:08.054734 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb975ad-cfd4-4d2f-aef2-3d15aa9af56c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:08.055578 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb975ad-cfd4-4d2f-aef2-3d15aa9af56c service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:08.056004 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb975ad-cfd4-4d2f-aef2-3d15aa9af56c service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:08.056444 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb975ad-cfd4-4d2f-aef2-3d15aa9af56c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:08.067077 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb975ad-cfd4-4d2f-aef2-3d15aa9af56c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:08.068499 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb975ad-cfd4-4d2f-aef2-3d15aa9af56c service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:08.078757 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1044/2072] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:47:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 13:47:08.368373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a562f9c8-ce4d-4f47-bfbc-b010279feb49 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:08.369919 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a562f9c8-ce4d-4f47-bfbc-b010279feb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:08.385593 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a562f9c8-ce4d-4f47-bfbc-b010279feb49 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:08.390240 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba012139-b0fe-4b4d-852a-ae586e4a586c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:08.392116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba012139-b0fe-4b4d-852a-ae586e4a586c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:08.392403 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba012139-b0fe-4b4d-852a-ae586e4a586c admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:08.392681 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba012139-b0fe-4b4d-852a-ae586e4a586c admin admin] PATH_INFO: `/v3/projects/a4acbd65fe8b458fb0d72843a3c8483c` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:08.394067 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba012139-b0fe-4b4d-852a-ae586e4a586c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4acbd65fe8b458fb0d72843a3c8483c)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:08.397210 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba012139-b0fe-4b4d-852a-ae586e4a586c admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:08.441523 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:08.445956 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-ba012139-b0fe-4b4d-852a-ae586e4a586c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4acbd65fe8b458fb0d72843a3c8483c', 'request_id': 'req-ba012139-b0fe-4b4d-852a-ae586e4a586c'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:08.453475 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-ba012139-b0fe-4b4d-852a-ae586e4a586c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4acbd65fe8b458fb0d72843a3c8483c', 'request_id': 'req-ba012139-b0fe-4b4d-852a-ae586e4a586c'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:08.462139 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-ba012139-b0fe-4b4d-852a-ae586e4a586c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4acbd65fe8b458fb0d72843a3c8483c', 'request_id': 'req-ba012139-b0fe-4b4d-852a-ae586e4a586c'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:08.473882 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-ba012139-b0fe-4b4d-852a-ae586e4a586c admin admin] The token cache is being invalidate because project a4acbd65fe8b458fb0d72843a3c8483c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:47:08.474168 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-ba012139-b0fe-4b4d-852a-ae586e4a586c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:08.474783 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-ba012139-b0fe-4b4d-852a-ae586e4a586c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:08.476361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1029/2073] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:47:08 2022] DELETE /identity/v3/projects/a4acbd65fe8b458fb0d72843a3c8483c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:08.480266 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb975ad-cfd4-4d2f-aef2-3d15aa9af56c service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:08.489899 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb975ad-cfd4-4d2f-aef2-3d15aa9af56c service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:08.548515 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca97d367-4c46-4e59-ae25-12a018e9edd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:08.549362 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97d367-4c46-4e59-ae25-12a018e9edd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:08.549649 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97d367-4c46-4e59-ae25-12a018e9edd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:08.549940 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97d367-4c46-4e59-ae25-12a018e9edd1 admin admin] PATH_INFO: `/v3/users/b37120b3f2ff489095fabeb7f7526c34` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:08.551877 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca97d367-4c46-4e59-ae25-12a018e9edd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b37120b3f2ff489095fabeb7f7526c34)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:08.553768 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca97d367-4c46-4e59-ae25-12a018e9edd1 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:08.603201 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-ca97d367-4c46-4e59-ae25-12a018e9edd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b37120b3f2ff489095fabeb7f7526c34'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:08.609931 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-ca97d367-4c46-4e59-ae25-12a018e9edd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b37120b3f2ff489095fabeb7f7526c34'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:08.617321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-ca97d367-4c46-4e59-ae25-12a018e9edd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b37120b3f2ff489095fabeb7f7526c34'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:08.617966 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-ca97d367-4c46-4e59-ae25-12a018e9edd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b37120b3f2ff489095fabeb7f7526c34'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:08.630555 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-ca97d367-4c46-4e59-ae25-12a018e9edd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b37120b3f2ff489095fabeb7f7526c34'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:08.632604 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:08.637106 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1045/2074] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:47:08 2022] DELETE /identity/v3/users/b37120b3f2ff489095fabeb7f7526c34 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:08.914735 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba012139-b0fe-4b4d-852a-ae586e4a586c admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:08.916197 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-ba012139-b0fe-4b4d-852a-ae586e4a586c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:08.930454 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba012139-b0fe-4b4d-852a-ae586e4a586c admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:09.004011 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b8212-03c4-4a04-83eb-36f99c485cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:09.005537 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b8212-03c4-4a04-83eb-36f99c485cd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:09.005821 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b8212-03c4-4a04-83eb-36f99c485cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:09.006097 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b8212-03c4-4a04-83eb-36f99c485cd6 admin admin] PATH_INFO: `/v3/projects/3661728af2cd45ea81fd134b63f17a67` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:09.007279 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637b8212-03c4-4a04-83eb-36f99c485cd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3661728af2cd45ea81fd134b63f17a67)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:09.009221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637b8212-03c4-4a04-83eb-36f99c485cd6 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:09.057203 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-637b8212-03c4-4a04-83eb-36f99c485cd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3661728af2cd45ea81fd134b63f17a67', 'request_id': 'req-637b8212-03c4-4a04-83eb-36f99c485cd6'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:09.063212 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-637b8212-03c4-4a04-83eb-36f99c485cd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3661728af2cd45ea81fd134b63f17a67', 'request_id': 'req-637b8212-03c4-4a04-83eb-36f99c485cd6'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:09.070680 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-637b8212-03c4-4a04-83eb-36f99c485cd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3661728af2cd45ea81fd134b63f17a67', 'request_id': 'req-637b8212-03c4-4a04-83eb-36f99c485cd6'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:09.080953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-637b8212-03c4-4a04-83eb-36f99c485cd6 admin admin] The token cache is being invalidate because project 3661728af2cd45ea81fd134b63f17a67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:47:09.081285 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-637b8212-03c4-4a04-83eb-36f99c485cd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:09.081920 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-637b8212-03c4-4a04-83eb-36f99c485cd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:09.083471 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1030/2075] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:47:08 2022] DELETE /identity/v3/projects/3661728af2cd45ea81fd134b63f17a67 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:09.088516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093043bb-6fce-448f-ae77-5e3a2031c39f None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:09.088987 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093043bb-6fce-448f-ae77-5e3a2031c39f None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:09.089099 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093043bb-6fce-448f-ae77-5e3a2031c39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:09.106855 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-093043bb-6fce-448f-ae77-5e3a2031c39f None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:09.108044 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-093043bb-6fce-448f-ae77-5e3a2031c39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:09.171095 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1046/2076] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:09.179911 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68cc36a-16b4-48df-8775-bb26fe242560 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:09.180074 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68cc36a-16b4-48df-8775-bb26fe242560 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:09.180357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68cc36a-16b4-48df-8775-bb26fe242560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:09.200633 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-b68cc36a-16b4-48df-8775-bb26fe242560 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:09.202023 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-b68cc36a-16b4-48df-8775-bb26fe242560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:09.213036 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1031/2077] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:09.215631 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093043bb-6fce-448f-ae77-5e3a2031c39f None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:09.227036 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093043bb-6fce-448f-ae77-5e3a2031c39f None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:09.233130 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8892634a-ed46-4e7b-adf6-6e4b426dbf40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:09.233130 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892634a-ed46-4e7b-adf6-6e4b426dbf40 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:09.233130 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892634a-ed46-4e7b-adf6-6e4b426dbf40 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:09.233456 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892634a-ed46-4e7b-adf6-6e4b426dbf40 admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:09.234274 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8892634a-ed46-4e7b-adf6-6e4b426dbf40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:47:09.234660 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8892634a-ed46-4e7b-adf6-6e4b426dbf40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:09.236458 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8892634a-ed46-4e7b-adf6-6e4b426dbf40 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:09.254500 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68cc36a-16b4-48df-8775-bb26fe242560 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:09.260840 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1047/2078] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:47:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:47:09.269644 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68cc36a-16b4-48df-8775-bb26fe242560 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:09.275192 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683bc4ba-c11b-486a-a02a-1fe3322a9ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:09.275939 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683bc4ba-c11b-486a-a02a-1fe3322a9ab7 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:09.276296 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683bc4ba-c11b-486a-a02a-1fe3322a9ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:09.276640 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683bc4ba-c11b-486a-a02a-1fe3322a9ab7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:09.277464 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683bc4ba-c11b-486a-a02a-1fe3322a9ab7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:09.279699 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683bc4ba-c11b-486a-a02a-1fe3322a9ab7 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:09.333183 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1032/2079] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:47:09 2022] POST /identity/v3/projects => generated 383 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:09.337532 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8892634a-ed46-4e7b-adf6-6e4b426dbf40 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:09.350179 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8892634a-ed46-4e7b-adf6-6e4b426dbf40 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:09.439605 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d656b9-7980-4fd3-90e8-80b6358098b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:09.442255 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d656b9-7980-4fd3-90e8-80b6358098b5 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:09.443407 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d656b9-7980-4fd3-90e8-80b6358098b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:09.443912 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d656b9-7980-4fd3-90e8-80b6358098b5 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:09.445999 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d656b9-7980-4fd3-90e8-80b6358098b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:09.450199 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d656b9-7980-4fd3-90e8-80b6358098b5 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:09.491832 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1048/2080] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:47:09 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:09.497581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683bc4ba-c11b-486a-a02a-1fe3322a9ab7 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:09.521822 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683bc4ba-c11b-486a-a02a-1fe3322a9ab7 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:09.541985 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561ea0e-3f2f-4a18-9f8d-306fca7cfe9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:09.543666 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561ea0e-3f2f-4a18-9f8d-306fca7cfe9f admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:09.545235 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561ea0e-3f2f-4a18-9f8d-306fca7cfe9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:09.545707 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561ea0e-3f2f-4a18-9f8d-306fca7cfe9f admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:09.546838 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7561ea0e-3f2f-4a18-9f8d-306fca7cfe9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:09.548806 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7561ea0e-3f2f-4a18-9f8d-306fca7cfe9f admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:09.573441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d656b9-7980-4fd3-90e8-80b6358098b5 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:09.585304 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1033/2081] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:09 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:47:09.616596 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d656b9-7980-4fd3-90e8-80b6358098b5 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:09.633879 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a7c6d-d430-4452-b032-5ff350bb2bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:09.637966 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a7c6d-d430-4452-b032-5ff350bb2bdf admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:09.638450 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a7c6d-d430-4452-b032-5ff350bb2bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:09.638823 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a7c6d-d430-4452-b032-5ff350bb2bdf admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:09.639849 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958a7c6d-d430-4452-b032-5ff350bb2bdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:09.644145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958a7c6d-d430-4452-b032-5ff350bb2bdf admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:09.662557 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1049/2082] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:09 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:47:09.666677 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561ea0e-3f2f-4a18-9f8d-306fca7cfe9f admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:09.683739 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561ea0e-3f2f-4a18-9f8d-306fca7cfe9f admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:09.690902 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcad6a9-51e5-4b43-b1aa-b9392963bcac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:09.693724 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcad6a9-51e5-4b43-b1aa-b9392963bcac admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:09.694319 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcad6a9-51e5-4b43-b1aa-b9392963bcac admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:09.695032 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcad6a9-51e5-4b43-b1aa-b9392963bcac admin admin] PATH_INFO: `/v3/projects/bda3e82b765a470099e45a1fc54435fb/users/b4feeb6e0adb47459fd2204d7cb2bb8c/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:09.716797 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcad6a9-51e5-4b43-b1aa-b9392963bcac admin admin] RBAC: Authorizing `identity:create_grant(project_id=bda3e82b765a470099e45a1fc54435fb, user_id=b4feeb6e0adb47459fd2204d7cb2bb8c, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:09.719853 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcad6a9-51e5-4b43-b1aa-b9392963bcac admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:09.730049 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:09.741077 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1034/2083] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:47:09 2022] PUT /identity/v3/projects/bda3e82b765a470099e45a1fc54435fb/users/b4feeb6e0adb47459fd2204d7cb2bb8c/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:09.746522 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f95cc5-3316-4618-81d4-4e27191d7037 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:09.746817 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f95cc5-3316-4618-81d4-4e27191d7037 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:09.747099 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f95cc5-3316-4618-81d4-4e27191d7037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:09.774010 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-54f95cc5-3316-4618-81d4-4e27191d7037 None None] MFA Rules not processed for user `b4feeb6e0adb47459fd2204d7cb2bb8c`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:09.775291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-54f95cc5-3316-4618-81d4-4e27191d7037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:09.831448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:09.842764 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1050/2084] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:09.847229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcad6a9-51e5-4b43-b1aa-b9392963bcac admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:09.912349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcad6a9-51e5-4b43-b1aa-b9392963bcac admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:09.917097 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5702fb1d-d67f-4a90-8769-42d3b577cd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:09.917755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702fb1d-d67f-4a90-8769-42d3b577cd05 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:09.918039 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702fb1d-d67f-4a90-8769-42d3b577cd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:09.918316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702fb1d-d67f-4a90-8769-42d3b577cd05 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:09.919018 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5702fb1d-d67f-4a90-8769-42d3b577cd05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:09.920920 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5702fb1d-d67f-4a90-8769-42d3b577cd05 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:09.951530 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1035/2085] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:47:09 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:09.952662 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f95cc5-3316-4618-81d4-4e27191d7037 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:09.964346 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f95cc5-3316-4618-81d4-4e27191d7037 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:10.036642 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5cc9a-a676-4e3a-be8d-cc78c9011c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:10.037391 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5cc9a-a676-4e3a-be8d-cc78c9011c4e admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:10.037709 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5cc9a-a676-4e3a-be8d-cc78c9011c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:10.038024 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5cc9a-a676-4e3a-be8d-cc78c9011c4e admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:10.038810 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5cc9a-a676-4e3a-be8d-cc78c9011c4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:10.041157 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5cc9a-a676-4e3a-be8d-cc78c9011c4e admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:10.067547 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1051/2086] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:47:09 2022] POST /identity/v3/users => generated 298 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:10.070216 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5702fb1d-d67f-4a90-8769-42d3b577cd05 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:10.080754 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5702fb1d-d67f-4a90-8769-42d3b577cd05 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:10.086268 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d944452-4c3b-486e-856c-b2ea21e22ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:10.086978 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d944452-4c3b-486e-856c-b2ea21e22ba5 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:10.087315 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d944452-4c3b-486e-856c-b2ea21e22ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:10.087616 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d944452-4c3b-486e-856c-b2ea21e22ba5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:10.088345 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d944452-4c3b-486e-856c-b2ea21e22ba5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:10.090068 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d944452-4c3b-486e-856c-b2ea21e22ba5 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:10.101328 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1036/2087] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:47:10.105376 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5cc9a-a676-4e3a-be8d-cc78c9011c4e admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:10.115409 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5cc9a-a676-4e3a-be8d-cc78c9011c4e admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:10.120015 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8081f2-d811-4908-a7df-947a7570eb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:10.120900 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8081f2-d811-4908-a7df-947a7570eb75 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:10.121219 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8081f2-d811-4908-a7df-947a7570eb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:10.121501 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8081f2-d811-4908-a7df-947a7570eb75 admin admin] PATH_INFO: `/v3/projects/6fe963545c9746c88b6aa7111429d91d/users/ce359d7dc6dc4c9e95683436fe4d2e48/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:10.136593 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8081f2-d811-4908-a7df-947a7570eb75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fe963545c9746c88b6aa7111429d91d, user_id=ce359d7dc6dc4c9e95683436fe4d2e48, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:10.139273 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8081f2-d811-4908-a7df-947a7570eb75 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:10.152780 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1052/2088] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:47:10 2022] PUT /identity/v3/projects/6fe963545c9746c88b6aa7111429d91d/users/ce359d7dc6dc4c9e95683436fe4d2e48/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:10.156773 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d944452-4c3b-486e-856c-b2ea21e22ba5 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:10.166571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d944452-4c3b-486e-856c-b2ea21e22ba5 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:10.224166 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a24c0-a2ea-4990-aa7c-225d16024bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:10.224861 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a24c0-a2ea-4990-aa7c-225d16024bf4 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:10.225182 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a24c0-a2ea-4990-aa7c-225d16024bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:10.225460 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a24c0-a2ea-4990-aa7c-225d16024bf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:10.226200 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a24c0-a2ea-4990-aa7c-225d16024bf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:10.227789 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a24c0-a2ea-4990-aa7c-225d16024bf4 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:10.240560 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1037/2089] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:47:10.245108 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8081f2-d811-4908-a7df-947a7570eb75 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:10.258403 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8081f2-d811-4908-a7df-947a7570eb75 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:10.263501 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98c023-3cf9-4642-896c-b28cd43b717e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:10.264412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98c023-3cf9-4642-896c-b28cd43b717e admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:10.264697 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98c023-3cf9-4642-896c-b28cd43b717e admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:10.264990 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98c023-3cf9-4642-896c-b28cd43b717e admin admin] PATH_INFO: `/v3/projects/6fe963545c9746c88b6aa7111429d91d/users/ce359d7dc6dc4c9e95683436fe4d2e48/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:10.267167 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98c023-3cf9-4642-896c-b28cd43b717e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fe963545c9746c88b6aa7111429d91d, user_id=ce359d7dc6dc4c9e95683436fe4d2e48, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:10.269923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98c023-3cf9-4642-896c-b28cd43b717e admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:10.308122 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1053/2090] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:47:10 2022] PUT /identity/v3/projects/6fe963545c9746c88b6aa7111429d91d/users/ce359d7dc6dc4c9e95683436fe4d2e48/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:10.319763 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c0a05-1d59-4d99-a8e7-80801d986dd4 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:10.320070 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c0a05-1d59-4d99-a8e7-80801d986dd4 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:10.320428 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c0a05-1d59-4d99-a8e7-80801d986dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:10.377498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-048c0a05-1d59-4d99-a8e7-80801d986dd4 None None] MFA Rules not processed for user `ce359d7dc6dc4c9e95683436fe4d2e48`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:10.380292 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-048c0a05-1d59-4d99-a8e7-80801d986dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:10.452802 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:10.469110 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1038/2091] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:10.480951 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98c023-3cf9-4642-896c-b28cd43b717e admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:10.483365 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-ac98c023-3cf9-4642-896c-b28cd43b717e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:10.563877 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98c023-3cf9-4642-896c-b28cd43b717e admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:10.578111 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d967373-b817-482f-a3d5-5d7768d657c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:10.591252 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d967373-b817-482f-a3d5-5d7768d657c0 service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:10.591252 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d967373-b817-482f-a3d5-5d7768d657c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:10.591252 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d967373-b817-482f-a3d5-5d7768d657c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:10.596907 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d967373-b817-482f-a3d5-5d7768d657c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:10.598673 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d967373-b817-482f-a3d5-5d7768d657c0 service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:10.609643 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1054/2092] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:47:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:47:10.774704 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c0a05-1d59-4d99-a8e7-80801d986dd4 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:10.789717 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c0a05-1d59-4d99-a8e7-80801d986dd4 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:10.793373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1757ec7e-8e23-45f8-bea1-30f5d243bef7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:10.794190 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757ec7e-8e23-45f8-bea1-30f5d243bef7 service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:10.794473 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757ec7e-8e23-45f8-bea1-30f5d243bef7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:10.794749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757ec7e-8e23-45f8-bea1-30f5d243bef7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:10.854233 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1757ec7e-8e23-45f8-bea1-30f5d243bef7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:10.855991 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1757ec7e-8e23-45f8-bea1-30f5d243bef7 service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:10.865942 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1039/2093] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:47:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:47:11.049973 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d967373-b817-482f-a3d5-5d7768d657c0 service cinder] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:11.060341 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d967373-b817-482f-a3d5-5d7768d657c0 service cinder] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:11.065486 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2fa6b9-ce50-4690-b45e-66ce2e2bd53e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:11.066199 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2fa6b9-ce50-4690-b45e-66ce2e2bd53e service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:11.066511 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2fa6b9-ce50-4690-b45e-66ce2e2bd53e service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:11.067056 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2fa6b9-ce50-4690-b45e-66ce2e2bd53e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:11.068922 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-1f2fa6b9-ce50-4690-b45e-66ce2e2bd53e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:11.140500 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:11.177602 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2fa6b9-ce50-4690-b45e-66ce2e2bd53e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:11.179181 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2fa6b9-ce50-4690-b45e-66ce2e2bd53e service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:11.200774 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1055/2094] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:47:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:47:11.589053 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1757ec7e-8e23-45f8-bea1-30f5d243bef7 service cinder] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:11.688050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1757ec7e-8e23-45f8-bea1-30f5d243bef7 service cinder] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:11.696673 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c1cff-7d8b-4112-9f49-af3ab86b84d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:11.707336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c1cff-7d8b-4112-9f49-af3ab86b84d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:11.707887 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c1cff-7d8b-4112-9f49-af3ab86b84d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:11.708536 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c1cff-7d8b-4112-9f49-af3ab86b84d9 admin admin] PATH_INFO: `/v3/users/b4feeb6e0adb47459fd2204d7cb2bb8c` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:11.709870 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7c1cff-7d8b-4112-9f49-af3ab86b84d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4feeb6e0adb47459fd2204d7cb2bb8c)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:11.711831 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7c1cff-7d8b-4112-9f49-af3ab86b84d9 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:11.821345 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-6e7c1cff-7d8b-4112-9f49-af3ab86b84d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4feeb6e0adb47459fd2204d7cb2bb8c'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:11.837761 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-6e7c1cff-7d8b-4112-9f49-af3ab86b84d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4feeb6e0adb47459fd2204d7cb2bb8c'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:11.838815 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-6e7c1cff-7d8b-4112-9f49-af3ab86b84d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4feeb6e0adb47459fd2204d7cb2bb8c'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:11.854425 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-6e7c1cff-7d8b-4112-9f49-af3ab86b84d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4feeb6e0adb47459fd2204d7cb2bb8c'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:11.870933 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-6e7c1cff-7d8b-4112-9f49-af3ab86b84d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4feeb6e0adb47459fd2204d7cb2bb8c'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:11.880480 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1040/2095] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:47:11 2022] DELETE /identity/v3/users/b4feeb6e0adb47459fd2204d7cb2bb8c => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:11.885437 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd0e7c-b295-4633-bca7-39f5280a977c None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:11.885729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd0e7c-b295-4633-bca7-39f5280a977c None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:11.886010 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd0e7c-b295-4633-bca7-39f5280a977c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:11.931927 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-83dd0e7c-b295-4633-bca7-39f5280a977c None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:11.937439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-83dd0e7c-b295-4633-bca7-39f5280a977c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:12.110488 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1056/2096] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:12.127964 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c1cff-7d8b-4112-9f49-af3ab86b84d9 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:12.130463 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-6e7c1cff-7d8b-4112-9f49-af3ab86b84d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:12.150176 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:12.210926 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c1cff-7d8b-4112-9f49-af3ab86b84d9 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:12.216357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d9a7b-00a9-485c-bf11-c2e3cff60a8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:12.217061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d9a7b-00a9-485c-bf11-c2e3cff60a8f service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:12.217349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d9a7b-00a9-485c-bf11-c2e3cff60a8f service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:12.217624 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d9a7b-00a9-485c-bf11-c2e3cff60a8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:12.228271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2d9a7b-00a9-485c-bf11-c2e3cff60a8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:12.230007 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2d9a7b-00a9-485c-bf11-c2e3cff60a8f service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:12.241937 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1041/2097] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:47:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:47:12.616970 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd0e7c-b295-4633-bca7-39f5280a977c None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:12.618460 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-83dd0e7c-b295-4633-bca7-39f5280a977c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:12.631765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd0e7c-b295-4633-bca7-39f5280a977c None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:12.637130 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e7220-3bc4-4730-93bd-54ef30431f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:12.637990 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e7220-3bc4-4730-93bd-54ef30431f3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:12.638266 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e7220-3bc4-4730-93bd-54ef30431f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:12.638541 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e7220-3bc4-4730-93bd-54ef30431f3e admin admin] PATH_INFO: `/v3/projects/bda3e82b765a470099e45a1fc54435fb` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:12.639717 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996e7220-3bc4-4730-93bd-54ef30431f3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bda3e82b765a470099e45a1fc54435fb)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:12.643367 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996e7220-3bc4-4730-93bd-54ef30431f3e admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:12.694086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-996e7220-3bc4-4730-93bd-54ef30431f3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bda3e82b765a470099e45a1fc54435fb', 'request_id': 'req-996e7220-3bc4-4730-93bd-54ef30431f3e'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:12.699794 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-996e7220-3bc4-4730-93bd-54ef30431f3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bda3e82b765a470099e45a1fc54435fb', 'request_id': 'req-996e7220-3bc4-4730-93bd-54ef30431f3e'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:12.707932 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-996e7220-3bc4-4730-93bd-54ef30431f3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bda3e82b765a470099e45a1fc54435fb', 'request_id': 'req-996e7220-3bc4-4730-93bd-54ef30431f3e'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:12.718923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-996e7220-3bc4-4730-93bd-54ef30431f3e admin admin] The token cache is being invalidate because project bda3e82b765a470099e45a1fc54435fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:47:12.719237 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-996e7220-3bc4-4730-93bd-54ef30431f3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:12.719849 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-996e7220-3bc4-4730-93bd-54ef30431f3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:12.721405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1057/2098] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:47:12 2022] DELETE /identity/v3/projects/bda3e82b765a470099e45a1fc54435fb => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:12.725965 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d9a7b-00a9-485c-bf11-c2e3cff60a8f service neutron] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:12.735813 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d9a7b-00a9-485c-bf11-c2e3cff60a8f service neutron] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:12.792217 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a8ccb0-a737-4b59-b9dd-9d5d46352216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:12.793126 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8ccb0-a737-4b59-b9dd-9d5d46352216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:12.793410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8ccb0-a737-4b59-b9dd-9d5d46352216 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:12.793685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8ccb0-a737-4b59-b9dd-9d5d46352216 admin admin] PATH_INFO: `/v3/users/ce359d7dc6dc4c9e95683436fe4d2e48` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:12.794867 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a8ccb0-a737-4b59-b9dd-9d5d46352216 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce359d7dc6dc4c9e95683436fe4d2e48)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:12.796778 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a8ccb0-a737-4b59-b9dd-9d5d46352216 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:12.841866 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-e4a8ccb0-a737-4b59-b9dd-9d5d46352216 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce359d7dc6dc4c9e95683436fe4d2e48'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:12.851455 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-e4a8ccb0-a737-4b59-b9dd-9d5d46352216 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce359d7dc6dc4c9e95683436fe4d2e48'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:12.852213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-e4a8ccb0-a737-4b59-b9dd-9d5d46352216 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce359d7dc6dc4c9e95683436fe4d2e48'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:12.862598 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-e4a8ccb0-a737-4b59-b9dd-9d5d46352216 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce359d7dc6dc4c9e95683436fe4d2e48'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:12.874687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-e4a8ccb0-a737-4b59-b9dd-9d5d46352216 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce359d7dc6dc4c9e95683436fe4d2e48'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:12.881164 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1042/2099] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:47:12 2022] DELETE /identity/v3/users/ce359d7dc6dc4c9e95683436fe4d2e48 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:13.242043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e7220-3bc4-4730-93bd-54ef30431f3e admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:13.243657 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-996e7220-3bc4-4730-93bd-54ef30431f3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:13.265600 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e7220-3bc4-4730-93bd-54ef30431f3e admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:13.338319 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95951cf5-e7f6-4ea3-bae2-08df7c84ce29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:13.339216 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95951cf5-e7f6-4ea3-bae2-08df7c84ce29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:13.339506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95951cf5-e7f6-4ea3-bae2-08df7c84ce29 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:13.339783 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95951cf5-e7f6-4ea3-bae2-08df7c84ce29 admin admin] PATH_INFO: `/v3/projects/6fe963545c9746c88b6aa7111429d91d` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:13.350822 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95951cf5-e7f6-4ea3-bae2-08df7c84ce29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fe963545c9746c88b6aa7111429d91d)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:13.350822 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95951cf5-e7f6-4ea3-bae2-08df7c84ce29 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:13.361631 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:13.396509 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-95951cf5-e7f6-4ea3-bae2-08df7c84ce29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fe963545c9746c88b6aa7111429d91d', 'request_id': 'req-95951cf5-e7f6-4ea3-bae2-08df7c84ce29'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:13.401989 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-95951cf5-e7f6-4ea3-bae2-08df7c84ce29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fe963545c9746c88b6aa7111429d91d', 'request_id': 'req-95951cf5-e7f6-4ea3-bae2-08df7c84ce29'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:13.409537 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-95951cf5-e7f6-4ea3-bae2-08df7c84ce29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fe963545c9746c88b6aa7111429d91d', 'request_id': 'req-95951cf5-e7f6-4ea3-bae2-08df7c84ce29'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:13.420887 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-95951cf5-e7f6-4ea3-bae2-08df7c84ce29 admin admin] The token cache is being invalidate because project 6fe963545c9746c88b6aa7111429d91d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:47:13.421481 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-95951cf5-e7f6-4ea3-bae2-08df7c84ce29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:13.421907 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-95951cf5-e7f6-4ea3-bae2-08df7c84ce29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:13.423643 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1058/2100] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:47:13 2022] DELETE /identity/v3/projects/6fe963545c9746c88b6aa7111429d91d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:13.429712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed760d25-316e-46f7-8cc6-bfae33f5c759 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:13.430031 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed760d25-316e-46f7-8cc6-bfae33f5c759 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:13.430337 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed760d25-316e-46f7-8cc6-bfae33f5c759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:13.452413 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-ed760d25-316e-46f7-8cc6-bfae33f5c759 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:13.453645 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-ed760d25-316e-46f7-8cc6-bfae33f5c759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:13.487389 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:13.520490 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1043/2101] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:47:13.530074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0dfa5-5098-42aa-803c-f4e9483c5ef6 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:13.530457 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0dfa5-5098-42aa-803c-f4e9483c5ef6 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:13.530697 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0dfa5-5098-42aa-803c-f4e9483c5ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:13.551963 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-aeb0dfa5-5098-42aa-803c-f4e9483c5ef6 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:13.553795 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-aeb0dfa5-5098-42aa-803c-f4e9483c5ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:13.565040 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1059/2102] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:13.568985 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed760d25-316e-46f7-8cc6-bfae33f5c759 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:13.578958 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed760d25-316e-46f7-8cc6-bfae33f5c759 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:13.585028 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2156cd-3f24-4d6d-8cf5-c9ba85bb6a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:13.585755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2156cd-3f24-4d6d-8cf5-c9ba85bb6a87 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:13.586041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2156cd-3f24-4d6d-8cf5-c9ba85bb6a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:13.586318 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2156cd-3f24-4d6d-8cf5-c9ba85bb6a87 admin admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:13.587396 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2156cd-3f24-4d6d-8cf5-c9ba85bb6a87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:47:13.587713 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2156cd-3f24-4d6d-8cf5-c9ba85bb6a87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:13.589467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2156cd-3f24-4d6d-8cf5-c9ba85bb6a87 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:13.612285 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1044/2103] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:47:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:47:13.614178 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0dfa5-5098-42aa-803c-f4e9483c5ef6 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:13.624334 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0dfa5-5098-42aa-803c-f4e9483c5ef6 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:13.640206 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd6926-d99a-4290-bdf2-4946a8626f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:13.640954 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd6926-d99a-4290-bdf2-4946a8626f2a admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:13.641338 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd6926-d99a-4290-bdf2-4946a8626f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:13.642199 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd6926-d99a-4290-bdf2-4946a8626f2a admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:13.642995 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cd6926-d99a-4290-bdf2-4946a8626f2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:13.645659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cd6926-d99a-4290-bdf2-4946a8626f2a admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:13.685326 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1060/2104] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:47:13 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:13.689426 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2156cd-3f24-4d6d-8cf5-c9ba85bb6a87 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:13.699995 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2156cd-3f24-4d6d-8cf5-c9ba85bb6a87 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:13.753571 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd6926-d99a-4290-bdf2-4946a8626f2a admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:13.769889 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-08cd6926-d99a-4290-bdf2-4946a8626f2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:13.774153 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13171508-dfb6-4c35-a049-8dc8087cb41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:13.775441 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13171508-dfb6-4c35-a049-8dc8087cb41b admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:13.775735 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13171508-dfb6-4c35-a049-8dc8087cb41b admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:13.776012 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13171508-dfb6-4c35-a049-8dc8087cb41b admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:13.777047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13171508-dfb6-4c35-a049-8dc8087cb41b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:13.780810 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13171508-dfb6-4c35-a049-8dc8087cb41b admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:13.784735 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd6926-d99a-4290-bdf2-4946a8626f2a admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:13.793495 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89129fd7-ec96-4a32-b81a-e8256e216bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:13.795295 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89129fd7-ec96-4a32-b81a-e8256e216bb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:13.795693 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89129fd7-ec96-4a32-b81a-e8256e216bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:13.796299 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89129fd7-ec96-4a32-b81a-e8256e216bb5 admin admin] PATH_INFO: `/v3/users/2a47cdf4a7e541eabb1d5019a2cff46a` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:13.797846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89129fd7-ec96-4a32-b81a-e8256e216bb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a47cdf4a7e541eabb1d5019a2cff46a)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:13.801640 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89129fd7-ec96-4a32-b81a-e8256e216bb5 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:13.807735 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1045/2105] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:47:13 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:13.811395 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13171508-dfb6-4c35-a049-8dc8087cb41b admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:13.822990 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13171508-dfb6-4c35-a049-8dc8087cb41b admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:13.828293 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c651451-7534-4765-bc0b-5cf9ebbc10bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:13.829088 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c651451-7534-4765-bc0b-5cf9ebbc10bc admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:13.829442 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c651451-7534-4765-bc0b-5cf9ebbc10bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:13.829788 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c651451-7534-4765-bc0b-5cf9ebbc10bc admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:13.830624 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c651451-7534-4765-bc0b-5cf9ebbc10bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:13.832949 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c651451-7534-4765-bc0b-5cf9ebbc10bc admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:13.845464 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1046/2106] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:47:13.850324 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c651451-7534-4765-bc0b-5cf9ebbc10bc admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:13.860711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c651451-7534-4765-bc0b-5cf9ebbc10bc admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:13.865761 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-89129fd7-ec96-4a32-b81a-e8256e216bb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a47cdf4a7e541eabb1d5019a2cff46a'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:13.865867 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d880e-843f-41bf-ba3f-1f527a415a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:13.866407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d880e-843f-41bf-ba3f-1f527a415a2b admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:13.866692 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d880e-843f-41bf-ba3f-1f527a415a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:13.866995 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d880e-843f-41bf-ba3f-1f527a415a2b admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:13.867731 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d880e-843f-41bf-ba3f-1f527a415a2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:13.869413 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d880e-843f-41bf-ba3f-1f527a415a2b admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:13.873509 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-89129fd7-ec96-4a32-b81a-e8256e216bb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a47cdf4a7e541eabb1d5019a2cff46a'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:13.881857 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1047/2107] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:47:13.882706 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-89129fd7-ec96-4a32-b81a-e8256e216bb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a47cdf4a7e541eabb1d5019a2cff46a'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:13.883425 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-89129fd7-ec96-4a32-b81a-e8256e216bb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a47cdf4a7e541eabb1d5019a2cff46a'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:13.886715 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d880e-843f-41bf-ba3f-1f527a415a2b admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:13.888436 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-879d880e-843f-41bf-ba3f-1f527a415a2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:13.896987 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-89129fd7-ec96-4a32-b81a-e8256e216bb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a47cdf4a7e541eabb1d5019a2cff46a'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:13.901898 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d880e-843f-41bf-ba3f-1f527a415a2b admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:13.903011 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1061/2108] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:47:13 2022] DELETE /identity/v3/users/2a47cdf4a7e541eabb1d5019a2cff46a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:47:13.909766 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f33551-9ae2-404a-b834-7f3f69fed091 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:13.910106 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f33551-9ae2-404a-b834-7f3f69fed091 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:13.910400 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f33551-9ae2-404a-b834-7f3f69fed091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:13.929019 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-42f33551-9ae2-404a-b834-7f3f69fed091 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:13.930230 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-42f33551-9ae2-404a-b834-7f3f69fed091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:13.972029 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e24212-58d0-4023-8005-d0b8741d5901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:13.972928 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e24212-58d0-4023-8005-d0b8741d5901 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:13.973517 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e24212-58d0-4023-8005-d0b8741d5901 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:13.973962 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e24212-58d0-4023-8005-d0b8741d5901 admin admin] PATH_INFO: `/v3/projects/988aa0dc994f46278d89ce396678323f/users/b4b67922ae904e97823865ca42b2649e/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:13.981197 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:13.992954 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e24212-58d0-4023-8005-d0b8741d5901 admin admin] RBAC: Authorizing `identity:create_grant(project_id=988aa0dc994f46278d89ce396678323f, user_id=b4b67922ae904e97823865ca42b2649e, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:13.995781 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e24212-58d0-4023-8005-d0b8741d5901 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:14.007309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1062/2109] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:47:14.010638 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1048/2110] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:47:13 2022] PUT /identity/v3/projects/988aa0dc994f46278d89ce396678323f/users/b4b67922ae904e97823865ca42b2649e/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:14.020808 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f33551-9ae2-404a-b834-7f3f69fed091 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:14.021695 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990cedb-6dd8-4954-815b-05368bf78b3d None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:14.022143 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990cedb-6dd8-4954-815b-05368bf78b3d None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:14.022269 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-42f33551-9ae2-404a-b834-7f3f69fed091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:14.022363 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990cedb-6dd8-4954-815b-05368bf78b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:14.046825 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:14.050174 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-9990cedb-6dd8-4954-815b-05368bf78b3d None None] MFA Rules not processed for user `b4b67922ae904e97823865ca42b2649e`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:14.052953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-9990cedb-6dd8-4954-815b-05368bf78b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:14.083369 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f33551-9ae2-404a-b834-7f3f69fed091 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:14.119526 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1049/2111] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:47:14.131420 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9990cedb-6dd8-4954-815b-05368bf78b3d None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:14.132942 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-9990cedb-6dd8-4954-815b-05368bf78b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:14.134760 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83788ce-eb7a-4fd9-ba3d-c3a38b625b0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:14.135414 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83788ce-eb7a-4fd9-ba3d-c3a38b625b0e service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:14.135700 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83788ce-eb7a-4fd9-ba3d-c3a38b625b0e service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:14.135980 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83788ce-eb7a-4fd9-ba3d-c3a38b625b0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:14.189912 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9990cedb-6dd8-4954-815b-05368bf78b3d None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:14.194632 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e155a8-4c05-40dd-b986-90b7a6c243b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:14.195284 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e155a8-4c05-40dd-b986-90b7a6c243b1 service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:14.195568 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e155a8-4c05-40dd-b986-90b7a6c243b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:14.195845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e155a8-4c05-40dd-b986-90b7a6c243b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:14.196478 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83788ce-eb7a-4fd9-ba3d-c3a38b625b0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:14.197939 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83788ce-eb7a-4fd9-ba3d-c3a38b625b0e service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:14.204199 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e155a8-4c05-40dd-b986-90b7a6c243b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:14.205645 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e155a8-4c05-40dd-b986-90b7a6c243b1 service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:14.208003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1063/2112] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:47:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 13:47:14.215146 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1050/2113] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:47:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 13:47:14.671718 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83788ce-eb7a-4fd9-ba3d-c3a38b625b0e service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:14.673355 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-e83788ce-eb7a-4fd9-ba3d-c3a38b625b0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:14.688007 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83788ce-eb7a-4fd9-ba3d-c3a38b625b0e service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:14.693178 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe0673-30fb-4ec9-9569-487ec14b62da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:14.694103 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe0673-30fb-4ec9-9569-487ec14b62da admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:14.694331 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe0673-30fb-4ec9-9569-487ec14b62da admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:14.694609 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe0673-30fb-4ec9-9569-487ec14b62da admin admin] PATH_INFO: `/v3/projects/fc641a408c23442e8d5d89c672e00c7a` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:14.695863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbe0673-30fb-4ec9-9569-487ec14b62da admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc641a408c23442e8d5d89c672e00c7a)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:14.697850 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbe0673-30fb-4ec9-9569-487ec14b62da admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:14.744351 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:14.746230 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-3dbe0673-30fb-4ec9-9569-487ec14b62da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc641a408c23442e8d5d89c672e00c7a', 'request_id': 'req-3dbe0673-30fb-4ec9-9569-487ec14b62da'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:14.755842 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-3dbe0673-30fb-4ec9-9569-487ec14b62da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc641a408c23442e8d5d89c672e00c7a', 'request_id': 'req-3dbe0673-30fb-4ec9-9569-487ec14b62da'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:14.766074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-3dbe0673-30fb-4ec9-9569-487ec14b62da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc641a408c23442e8d5d89c672e00c7a', 'request_id': 'req-3dbe0673-30fb-4ec9-9569-487ec14b62da'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:14.779271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-3dbe0673-30fb-4ec9-9569-487ec14b62da admin admin] The token cache is being invalidate because project fc641a408c23442e8d5d89c672e00c7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:47:14.779557 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-3dbe0673-30fb-4ec9-9569-487ec14b62da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:14.780196 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-3dbe0673-30fb-4ec9-9569-487ec14b62da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:14.781775 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1064/2114] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:47:14 2022] DELETE /identity/v3/projects/fc641a408c23442e8d5d89c672e00c7a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:14.786330 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e155a8-4c05-40dd-b986-90b7a6c243b1 service cinder] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:14.797629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e155a8-4c05-40dd-b986-90b7a6c243b1 service cinder] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:14.859725 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe7581-f3c3-48a6-b427-5d71ff63d231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:14.859725 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe7581-f3c3-48a6-b427-5d71ff63d231 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:14.859725 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe7581-f3c3-48a6-b427-5d71ff63d231 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:14.860060 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe7581-f3c3-48a6-b427-5d71ff63d231 admin admin] PATH_INFO: `/v3/users/20be1661539244d389426433cd192e65` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:14.861040 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe7581-f3c3-48a6-b427-5d71ff63d231 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20be1661539244d389426433cd192e65)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:14.862956 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe7581-f3c3-48a6-b427-5d71ff63d231 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:14.927366 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-d5fe7581-f3c3-48a6-b427-5d71ff63d231 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20be1661539244d389426433cd192e65'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:14.929129 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:14.935871 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-d5fe7581-f3c3-48a6-b427-5d71ff63d231 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20be1661539244d389426433cd192e65'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:14.936682 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-d5fe7581-f3c3-48a6-b427-5d71ff63d231 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20be1661539244d389426433cd192e65'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:14.944318 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-d5fe7581-f3c3-48a6-b427-5d71ff63d231 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20be1661539244d389426433cd192e65'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:14.956427 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-d5fe7581-f3c3-48a6-b427-5d71ff63d231 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20be1661539244d389426433cd192e65'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:14.964405 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1051/2115] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:47:14 2022] DELETE /identity/v3/users/20be1661539244d389426433cd192e65 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:47:15.267126 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe0673-30fb-4ec9-9569-487ec14b62da admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:15.268644 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-3dbe0673-30fb-4ec9-9569-487ec14b62da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:15.284424 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe0673-30fb-4ec9-9569-487ec14b62da admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:15.369250 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4644e5c8-9768-4d38-95e1-38d0ded29255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:15.370091 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644e5c8-9768-4d38-95e1-38d0ded29255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:15.370365 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644e5c8-9768-4d38-95e1-38d0ded29255 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:15.370644 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644e5c8-9768-4d38-95e1-38d0ded29255 admin admin] PATH_INFO: `/v3/projects/3a1c0eb8675f468a9a06f4ea8969437c` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:15.371837 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4644e5c8-9768-4d38-95e1-38d0ded29255 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a1c0eb8675f468a9a06f4ea8969437c)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:15.373835 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4644e5c8-9768-4d38-95e1-38d0ded29255 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:15.421952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-4644e5c8-9768-4d38-95e1-38d0ded29255 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a1c0eb8675f468a9a06f4ea8969437c', 'request_id': 'req-4644e5c8-9768-4d38-95e1-38d0ded29255'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:15.429223 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-4644e5c8-9768-4d38-95e1-38d0ded29255 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a1c0eb8675f468a9a06f4ea8969437c', 'request_id': 'req-4644e5c8-9768-4d38-95e1-38d0ded29255'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:15.436283 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-4644e5c8-9768-4d38-95e1-38d0ded29255 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a1c0eb8675f468a9a06f4ea8969437c', 'request_id': 'req-4644e5c8-9768-4d38-95e1-38d0ded29255'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:15.447723 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-4644e5c8-9768-4d38-95e1-38d0ded29255 admin admin] The token cache is being invalidate because project 3a1c0eb8675f468a9a06f4ea8969437c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:47:15.448013 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-4644e5c8-9768-4d38-95e1-38d0ded29255 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:15.448633 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-4644e5c8-9768-4d38-95e1-38d0ded29255 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:15.450199 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1065/2116] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:47:15 2022] DELETE /identity/v3/projects/3a1c0eb8675f468a9a06f4ea8969437c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:15.454153 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe7581-f3c3-48a6-b427-5d71ff63d231 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:15.455530 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-d5fe7581-f3c3-48a6-b427-5d71ff63d231 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:15.468003 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe7581-f3c3-48a6-b427-5d71ff63d231 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:15.533767 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130557e9-e4a5-4db3-8cf8-c2d3fd5d3032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:15.534890 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130557e9-e4a5-4db3-8cf8-c2d3fd5d3032 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:15.535189 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130557e9-e4a5-4db3-8cf8-c2d3fd5d3032 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:15.536284 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130557e9-e4a5-4db3-8cf8-c2d3fd5d3032 admin admin] PATH_INFO: `/v3/users/2c91cc24faa44fd68ddb05222106b67b` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:15.537663 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130557e9-e4a5-4db3-8cf8-c2d3fd5d3032 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c91cc24faa44fd68ddb05222106b67b)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:15.539633 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130557e9-e4a5-4db3-8cf8-c2d3fd5d3032 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:15.606555 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-130557e9-e4a5-4db3-8cf8-c2d3fd5d3032 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c91cc24faa44fd68ddb05222106b67b'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:15.616187 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-130557e9-e4a5-4db3-8cf8-c2d3fd5d3032 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c91cc24faa44fd68ddb05222106b67b'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:15.618311 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-130557e9-e4a5-4db3-8cf8-c2d3fd5d3032 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c91cc24faa44fd68ddb05222106b67b'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:15.632800 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-130557e9-e4a5-4db3-8cf8-c2d3fd5d3032 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c91cc24faa44fd68ddb05222106b67b'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:15.671043 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-130557e9-e4a5-4db3-8cf8-c2d3fd5d3032 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c91cc24faa44fd68ddb05222106b67b'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:15.685508 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1052/2117] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:47:15 2022] DELETE /identity/v3/users/2c91cc24faa44fd68ddb05222106b67b => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:16.179899 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4644e5c8-9768-4d38-95e1-38d0ded29255 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:16.179899 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-4644e5c8-9768-4d38-95e1-38d0ded29255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:16.207171 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4644e5c8-9768-4d38-95e1-38d0ded29255 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:16.282429 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffece7f4-85a6-4a65-b410-b4ee32cc0a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:16.283191 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffece7f4-85a6-4a65-b410-b4ee32cc0a49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:16.283473 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffece7f4-85a6-4a65-b410-b4ee32cc0a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:16.283747 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffece7f4-85a6-4a65-b410-b4ee32cc0a49 admin admin] PATH_INFO: `/v3/projects/65cfa71f869e42f6b3430149f7264129` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:16.284930 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffece7f4-85a6-4a65-b410-b4ee32cc0a49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65cfa71f869e42f6b3430149f7264129)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:16.287486 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffece7f4-85a6-4a65-b410-b4ee32cc0a49 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:16.301218 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:16.348620 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-ffece7f4-85a6-4a65-b410-b4ee32cc0a49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65cfa71f869e42f6b3430149f7264129', 'request_id': 'req-ffece7f4-85a6-4a65-b410-b4ee32cc0a49'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:16.354927 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-ffece7f4-85a6-4a65-b410-b4ee32cc0a49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65cfa71f869e42f6b3430149f7264129', 'request_id': 'req-ffece7f4-85a6-4a65-b410-b4ee32cc0a49'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:16.364072 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-ffece7f4-85a6-4a65-b410-b4ee32cc0a49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65cfa71f869e42f6b3430149f7264129', 'request_id': 'req-ffece7f4-85a6-4a65-b410-b4ee32cc0a49'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:16.376084 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-ffece7f4-85a6-4a65-b410-b4ee32cc0a49 admin admin] The token cache is being invalidate because project 65cfa71f869e42f6b3430149f7264129 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:47:16.376377 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-ffece7f4-85a6-4a65-b410-b4ee32cc0a49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:16.377085 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-ffece7f4-85a6-4a65-b410-b4ee32cc0a49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:16.378646 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1066/2118] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:47:16 2022] DELETE /identity/v3/projects/65cfa71f869e42f6b3430149f7264129 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:16.384588 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06710efd-7e0d-4942-956a-94373ea51bc8 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:16.385081 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06710efd-7e0d-4942-956a-94373ea51bc8 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:16.385212 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06710efd-7e0d-4942-956a-94373ea51bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:16.404733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-06710efd-7e0d-4942-956a-94373ea51bc8 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:16.405986 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-06710efd-7e0d-4942-956a-94373ea51bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:16.455532 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:16.481257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1053/2119] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:16.489630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7c0e6-9ac5-4c62-adef-5ae98ca5fc64 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:16.489913 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7c0e6-9ac5-4c62-adef-5ae98ca5fc64 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:16.490187 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7c0e6-9ac5-4c62-adef-5ae98ca5fc64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:16.510056 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-2dc7c0e6-9ac5-4c62-adef-5ae98ca5fc64 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:16.511327 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-2dc7c0e6-9ac5-4c62-adef-5ae98ca5fc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:16.519226 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1067/2120] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:16.524239 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06710efd-7e0d-4942-956a-94373ea51bc8 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:16.533963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06710efd-7e0d-4942-956a-94373ea51bc8 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:16.539348 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab30f9-d158-442d-ac73-fe934592a4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:16.540002 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab30f9-d158-442d-ac73-fe934592a4b5 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:16.540283 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab30f9-d158-442d-ac73-fe934592a4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:16.540555 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab30f9-d158-442d-ac73-fe934592a4b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:16.541337 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab30f9-d158-442d-ac73-fe934592a4b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:47:16.541641 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab30f9-d158-442d-ac73-fe934592a4b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:16.543233 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab30f9-d158-442d-ac73-fe934592a4b5 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:16.557364 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1054/2121] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:47:16.561960 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7c0e6-9ac5-4c62-adef-5ae98ca5fc64 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:16.571446 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7c0e6-9ac5-4c62-adef-5ae98ca5fc64 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:16.576348 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155e992-775d-4894-b407-876a178be0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:16.577166 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155e992-775d-4894-b407-876a178be0d3 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:16.577484 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155e992-775d-4894-b407-876a178be0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:16.577770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155e992-775d-4894-b407-876a178be0d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:16.578522 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155e992-775d-4894-b407-876a178be0d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:16.580455 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155e992-775d-4894-b407-876a178be0d3 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:16.610583 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1068/2122] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:47:16 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:47:16.615501 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab30f9-d158-442d-ac73-fe934592a4b5 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:16.625899 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab30f9-d158-442d-ac73-fe934592a4b5 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:16.693388 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863a424-6849-4642-93ca-d0d82bd5fa33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:16.694067 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863a424-6849-4642-93ca-d0d82bd5fa33 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:16.694351 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863a424-6849-4642-93ca-d0d82bd5fa33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:16.694625 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863a424-6849-4642-93ca-d0d82bd5fa33 admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:16.695343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863a424-6849-4642-93ca-d0d82bd5fa33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:16.697365 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863a424-6849-4642-93ca-d0d82bd5fa33 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:16.718591 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1055/2123] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:47:16 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:16.722561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155e992-775d-4894-b407-876a178be0d3 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:16.733764 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155e992-775d-4894-b407-876a178be0d3 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:16.738681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d2070-2d51-4bb6-8025-51fb0493a5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:16.739752 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d2070-2d51-4bb6-8025-51fb0493a5de admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:16.740102 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d2070-2d51-4bb6-8025-51fb0493a5de admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:16.740464 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d2070-2d51-4bb6-8025-51fb0493a5de admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:16.741390 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d2070-2d51-4bb6-8025-51fb0493a5de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:16.743092 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d2070-2d51-4bb6-8025-51fb0493a5de admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:16.757089 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1069/2124] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:47:16.760842 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863a424-6849-4642-93ca-d0d82bd5fa33 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:16.771758 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863a424-6849-4642-93ca-d0d82bd5fa33 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:16.776583 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb6ec2d-d67a-4ca8-9a88-14d9ad11a914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:16.777291 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb6ec2d-d67a-4ca8-9a88-14d9ad11a914 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:16.777580 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb6ec2d-d67a-4ca8-9a88-14d9ad11a914 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:16.777859 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb6ec2d-d67a-4ca8-9a88-14d9ad11a914 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:16.778595 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb6ec2d-d67a-4ca8-9a88-14d9ad11a914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:16.780208 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb6ec2d-d67a-4ca8-9a88-14d9ad11a914 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:16.791276 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1056/2125] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:47:16.795614 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d2070-2d51-4bb6-8025-51fb0493a5de admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:16.806786 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d2070-2d51-4bb6-8025-51fb0493a5de admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:16.814146 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb49d5-3681-4156-9869-c7fcdfeeb2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:16.815086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb49d5-3681-4156-9869-c7fcdfeeb2b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:16.815374 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb49d5-3681-4156-9869-c7fcdfeeb2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:16.815698 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb49d5-3681-4156-9869-c7fcdfeeb2b1 admin admin] PATH_INFO: `/v3/projects/36d69e9074de4434af0946d86c2f809d/users/a6499cf76cba42aa9ffe4689cfabcf66/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:16.833819 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb49d5-3681-4156-9869-c7fcdfeeb2b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36d69e9074de4434af0946d86c2f809d, user_id=a6499cf76cba42aa9ffe4689cfabcf66, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:16.835499 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb49d5-3681-4156-9869-c7fcdfeeb2b1 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:16.849544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1070/2126] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:47:16 2022] PUT /identity/v3/projects/36d69e9074de4434af0946d86c2f809d/users/a6499cf76cba42aa9ffe4689cfabcf66/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:16.861397 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc9a56-e646-4cb2-8e72-9caffe5ebbef None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:16.861721 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc9a56-e646-4cb2-8e72-9caffe5ebbef None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:16.861967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc9a56-e646-4cb2-8e72-9caffe5ebbef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:16.889400 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-4ffc9a56-e646-4cb2-8e72-9caffe5ebbef None None] MFA Rules not processed for user `a6499cf76cba42aa9ffe4689cfabcf66`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:16.890519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-4ffc9a56-e646-4cb2-8e72-9caffe5ebbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:16.955406 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1057/2127] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:16.960555 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb49d5-3681-4156-9869-c7fcdfeeb2b1 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:17.002108 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc9a56-e646-4cb2-8e72-9caffe5ebbef None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:17.003494 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-4ffc9a56-e646-4cb2-8e72-9caffe5ebbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:17.025222 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb49d5-3681-4156-9869-c7fcdfeeb2b1 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:17.030343 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24a91c-fbbd-43ee-afeb-a2ed5fb62ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:17.031005 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24a91c-fbbd-43ee-afeb-a2ed5fb62ac6 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:17.031289 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24a91c-fbbd-43ee-afeb-a2ed5fb62ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:17.031568 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24a91c-fbbd-43ee-afeb-a2ed5fb62ac6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:17.032269 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc24a91c-fbbd-43ee-afeb-a2ed5fb62ac6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:17.034207 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc24a91c-fbbd-43ee-afeb-a2ed5fb62ac6 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:17.050243 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:17.064151 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1071/2128] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:47:16 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:17.065208 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24a91c-fbbd-43ee-afeb-a2ed5fb62ac6 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:17.075870 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc9a56-e646-4cb2-8e72-9caffe5ebbef None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:17.087673 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:17.133798 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24a91c-fbbd-43ee-afeb-a2ed5fb62ac6 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:17.134021 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de286449-e8ee-4aa8-ba25-796c2a0c9a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:17.134907 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de286449-e8ee-4aa8-ba25-796c2a0c9a2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:17.135196 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de286449-e8ee-4aa8-ba25-796c2a0c9a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:17.135465 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de286449-e8ee-4aa8-ba25-796c2a0c9a2c admin admin] PATH_INFO: `/v3/users/b4b67922ae904e97823865ca42b2649e` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:17.136723 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de286449-e8ee-4aa8-ba25-796c2a0c9a2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4b67922ae904e97823865ca42b2649e)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:17.138644 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de286449-e8ee-4aa8-ba25-796c2a0c9a2c admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:17.138768 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026bff5-8e07-4bc0-91ac-fb3cf8f3493d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:17.139325 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026bff5-8e07-4bc0-91ac-fb3cf8f3493d admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:17.139615 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026bff5-8e07-4bc0-91ac-fb3cf8f3493d admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:17.139902 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026bff5-8e07-4bc0-91ac-fb3cf8f3493d admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:17.140630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026bff5-8e07-4bc0-91ac-fb3cf8f3493d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:17.142570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026bff5-8e07-4bc0-91ac-fb3cf8f3493d admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:17.163859 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1072/2129] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:47:17 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:47:17.168172 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026bff5-8e07-4bc0-91ac-fb3cf8f3493d admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:17.178404 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026bff5-8e07-4bc0-91ac-fb3cf8f3493d admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:17.183146 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b22745-7630-41b7-8da8-76c1766c85ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:17.183808 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b22745-7630-41b7-8da8-76c1766c85ea admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:17.184123 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b22745-7630-41b7-8da8-76c1766c85ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:17.184426 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b22745-7630-41b7-8da8-76c1766c85ea admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:17.185214 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b22745-7630-41b7-8da8-76c1766c85ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:17.186839 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b22745-7630-41b7-8da8-76c1766c85ea admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:17.190616 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-de286449-e8ee-4aa8-ba25-796c2a0c9a2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4b67922ae904e97823865ca42b2649e'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:17.198519 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1073/2130] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:47:17.198944 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-de286449-e8ee-4aa8-ba25-796c2a0c9a2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4b67922ae904e97823865ca42b2649e'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:17.199623 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-de286449-e8ee-4aa8-ba25-796c2a0c9a2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4b67922ae904e97823865ca42b2649e'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:17.202496 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b22745-7630-41b7-8da8-76c1766c85ea admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:17.205146 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-b3b22745-7630-41b7-8da8-76c1766c85ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:17.210978 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-de286449-e8ee-4aa8-ba25-796c2a0c9a2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4b67922ae904e97823865ca42b2649e'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:17.218310 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b22745-7630-41b7-8da8-76c1766c85ea admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:17.225676 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-de286449-e8ee-4aa8-ba25-796c2a0c9a2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4b67922ae904e97823865ca42b2649e'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:17.226595 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7197be-bfa6-47a2-8bbc-5919f3e24c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:17.228088 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7197be-bfa6-47a2-8bbc-5919f3e24c02 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:17.228523 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7197be-bfa6-47a2-8bbc-5919f3e24c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:17.228814 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7197be-bfa6-47a2-8bbc-5919f3e24c02 admin admin] PATH_INFO: `/v3/projects/9d6cc98e9115430f94da07fc567a48e7/users/b0116bcfeee144e0908e93b9a7661613/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:17.232462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1058/2131] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:47:17 2022] DELETE /identity/v3/users/b4b67922ae904e97823865ca42b2649e => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:47:17.236956 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25531d6e-cba5-4ebd-b611-536efd4dd913 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:17.238605 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25531d6e-cba5-4ebd-b611-536efd4dd913 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:17.238989 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25531d6e-cba5-4ebd-b611-536efd4dd913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:17.246890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7197be-bfa6-47a2-8bbc-5919f3e24c02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d6cc98e9115430f94da07fc567a48e7, user_id=b0116bcfeee144e0908e93b9a7661613, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:17.249602 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7197be-bfa6-47a2-8bbc-5919f3e24c02 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:17.259995 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-25531d6e-cba5-4ebd-b611-536efd4dd913 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:17.261519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-25531d6e-cba5-4ebd-b611-536efd4dd913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:17.263984 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1074/2132] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:47:17 2022] PUT /identity/v3/projects/9d6cc98e9115430f94da07fc567a48e7/users/b0116bcfeee144e0908e93b9a7661613/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:17.267890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7197be-bfa6-47a2-8bbc-5919f3e24c02 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:17.357471 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1059/2133] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:17.363974 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25531d6e-cba5-4ebd-b611-536efd4dd913 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:17.365450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-25531d6e-cba5-4ebd-b611-536efd4dd913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:17.369911 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7197be-bfa6-47a2-8bbc-5919f3e24c02 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:17.374931 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb5bc4-243b-4056-917e-f51bd0a19687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:17.375609 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb5bc4-243b-4056-917e-f51bd0a19687 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:17.375794 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:17.375928 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb5bc4-243b-4056-917e-f51bd0a19687 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:17.376217 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb5bc4-243b-4056-917e-f51bd0a19687 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:17.376994 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb5bc4-243b-4056-917e-f51bd0a19687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:17.378785 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb5bc4-243b-4056-917e-f51bd0a19687 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:17.390866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1075/2134] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:17 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:47:17.395647 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb5bc4-243b-4056-917e-f51bd0a19687 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:17.407815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb5bc4-243b-4056-917e-f51bd0a19687 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:17.413466 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b02ba18-59dd-4c18-b340-a0a478f5e2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:17.414540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b02ba18-59dd-4c18-b340-a0a478f5e2a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:17.414834 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b02ba18-59dd-4c18-b340-a0a478f5e2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:17.415126 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b02ba18-59dd-4c18-b340-a0a478f5e2a2 admin admin] PATH_INFO: `/v3/projects/9d6cc98e9115430f94da07fc567a48e7/users/b0116bcfeee144e0908e93b9a7661613/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:17.417197 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b02ba18-59dd-4c18-b340-a0a478f5e2a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d6cc98e9115430f94da07fc567a48e7, user_id=b0116bcfeee144e0908e93b9a7661613, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:17.419999 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b02ba18-59dd-4c18-b340-a0a478f5e2a2 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:17.436868 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1076/2135] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:47:17 2022] PUT /identity/v3/projects/9d6cc98e9115430f94da07fc567a48e7/users/b0116bcfeee144e0908e93b9a7661613/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:47:17.440606 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25531d6e-cba5-4ebd-b611-536efd4dd913 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:17.447158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a4d6d-44b0-4d0c-a3cc-6f583b5138ff None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:17.447449 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a4d6d-44b0-4d0c-a3cc-6f583b5138ff None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:17.447735 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a4d6d-44b0-4d0c-a3cc-6f583b5138ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:17.479164 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-748a4d6d-44b0-4d0c-a3cc-6f583b5138ff None None] MFA Rules not processed for user `b0116bcfeee144e0908e93b9a7661613`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:17.480514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-748a4d6d-44b0-4d0c-a3cc-6f583b5138ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:17.513755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebda7b4-2099-4b75-8d52-491e40e28df5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:17.514541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebda7b4-2099-4b75-8d52-491e40e28df5 service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:17.514910 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebda7b4-2099-4b75-8d52-491e40e28df5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:17.515291 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebda7b4-2099-4b75-8d52-491e40e28df5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:17.547489 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:17.584952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1077/2136] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:17 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:47:17.586472 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebda7b4-2099-4b75-8d52-491e40e28df5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:17.587905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebda7b4-2099-4b75-8d52-491e40e28df5 service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:17.595726 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a4d6d-44b0-4d0c-a3cc-6f583b5138ff None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:17.597159 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-748a4d6d-44b0-4d0c-a3cc-6f583b5138ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:17.599833 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1060/2137] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:47:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 13:47:17.659726 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a4d6d-44b0-4d0c-a3cc-6f583b5138ff None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:17.664318 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7ef9b-a0dd-4e0d-8101-b0fad630a78b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:17.665109 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7ef9b-a0dd-4e0d-8101-b0fad630a78b service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:17.665437 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7ef9b-a0dd-4e0d-8101-b0fad630a78b service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:17.665718 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7ef9b-a0dd-4e0d-8101-b0fad630a78b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:17.677601 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7ef9b-a0dd-4e0d-8101-b0fad630a78b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:17.679077 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7ef9b-a0dd-4e0d-8101-b0fad630a78b service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:17.688866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1078/2138] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:47:17 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 13:47:17.879246 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebda7b4-2099-4b75-8d52-491e40e28df5 service neutron] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:17.880785 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-8ebda7b4-2099-4b75-8d52-491e40e28df5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:17.896887 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebda7b4-2099-4b75-8d52-491e40e28df5 service neutron] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:17.901875 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf993168-f121-4ae3-aea6-6bcf15545795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:17.902890 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf993168-f121-4ae3-aea6-6bcf15545795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:17.903179 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf993168-f121-4ae3-aea6-6bcf15545795 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:17.903453 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf993168-f121-4ae3-aea6-6bcf15545795 admin admin] PATH_INFO: `/v3/projects/988aa0dc994f46278d89ce396678323f` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:17.904699 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf993168-f121-4ae3-aea6-6bcf15545795 admin admin] RBAC: Authorizing `identity:delete_project(project_id=988aa0dc994f46278d89ce396678323f)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:17.906622 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf993168-f121-4ae3-aea6-6bcf15545795 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:17.960389 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-bf993168-f121-4ae3-aea6-6bcf15545795 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '988aa0dc994f46278d89ce396678323f', 'request_id': 'req-bf993168-f121-4ae3-aea6-6bcf15545795'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:17.969132 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-bf993168-f121-4ae3-aea6-6bcf15545795 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '988aa0dc994f46278d89ce396678323f', 'request_id': 'req-bf993168-f121-4ae3-aea6-6bcf15545795'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:17.979158 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-bf993168-f121-4ae3-aea6-6bcf15545795 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '988aa0dc994f46278d89ce396678323f', 'request_id': 'req-bf993168-f121-4ae3-aea6-6bcf15545795'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:17.984438 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:17.997257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-bf993168-f121-4ae3-aea6-6bcf15545795 admin admin] The token cache is being invalidate because project 988aa0dc994f46278d89ce396678323f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:47:17.997763 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-bf993168-f121-4ae3-aea6-6bcf15545795 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:17.998708 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-bf993168-f121-4ae3-aea6-6bcf15545795 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:18.000566 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1061/2139] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:47:17 2022] DELETE /identity/v3/projects/988aa0dc994f46278d89ce396678323f => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:18.005849 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3421f8-a71d-4841-a0b3-9ff67ae96557 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:18.006142 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3421f8-a71d-4841-a0b3-9ff67ae96557 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:18.006426 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3421f8-a71d-4841-a0b3-9ff67ae96557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:18.027726 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-de3421f8-a71d-4841-a0b3-9ff67ae96557 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:18.029562 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-de3421f8-a71d-4841-a0b3-9ff67ae96557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:18.117041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1079/2140] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:18.122129 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66853805-65a8-4132-aa5d-caa6d13c5137 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:18.122423 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66853805-65a8-4132-aa5d-caa6d13c5137 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:18.122706 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66853805-65a8-4132-aa5d-caa6d13c5137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:18.143236 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-66853805-65a8-4132-aa5d-caa6d13c5137 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:18.144447 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-66853805-65a8-4132-aa5d-caa6d13c5137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:18.167403 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1062/2141] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:18.171769 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3421f8-a71d-4841-a0b3-9ff67ae96557 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:18.183820 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3421f8-a71d-4841-a0b3-9ff67ae96557 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:18.191060 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089f96e-b6c9-4b40-947f-20e770c07b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:18.192309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089f96e-b6c9-4b40-947f-20e770c07b82 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:18.192598 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089f96e-b6c9-4b40-947f-20e770c07b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:18.192876 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089f96e-b6c9-4b40-947f-20e770c07b82 admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:18.193820 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f089f96e-b6c9-4b40-947f-20e770c07b82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:47:18.194134 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f089f96e-b6c9-4b40-947f-20e770c07b82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:18.196362 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f089f96e-b6c9-4b40-947f-20e770c07b82 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:18.212532 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1080/2142] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:47:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:47:18.217198 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66853805-65a8-4132-aa5d-caa6d13c5137 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:18.229680 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66853805-65a8-4132-aa5d-caa6d13c5137 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:18.236859 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046612c7-139d-475f-a565-c740437b4fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:18.238541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046612c7-139d-475f-a565-c740437b4fb8 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:18.239408 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046612c7-139d-475f-a565-c740437b4fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:18.239760 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046612c7-139d-475f-a565-c740437b4fb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:18.240552 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046612c7-139d-475f-a565-c740437b4fb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:18.243986 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046612c7-139d-475f-a565-c740437b4fb8 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:18.280253 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1063/2143] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:47:18 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:18.281340 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089f96e-b6c9-4b40-947f-20e770c07b82 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:18.295741 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089f96e-b6c9-4b40-947f-20e770c07b82 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:18.384437 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f291b7a-d250-4ab9-8bab-7492d6a84a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:18.385549 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f291b7a-d250-4ab9-8bab-7492d6a84a9c admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:18.386140 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f291b7a-d250-4ab9-8bab-7492d6a84a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:18.386614 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f291b7a-d250-4ab9-8bab-7492d6a84a9c admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:18.387532 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f291b7a-d250-4ab9-8bab-7492d6a84a9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:18.389689 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f291b7a-d250-4ab9-8bab-7492d6a84a9c admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:18.395836 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046612c7-139d-475f-a565-c740437b4fb8 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:18.398488 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-046612c7-139d-475f-a565-c740437b4fb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:18.408646 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:18.415266 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046612c7-139d-475f-a565-c740437b4fb8 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:18.419900 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1081/2144] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:47:18 2022] POST /identity/v3/users => generated 293 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:18.423405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f291b7a-d250-4ab9-8bab-7492d6a84a9c admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:18.424202 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44f519-f5ed-4038-b51f-190bd94be1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:18.425413 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44f519-f5ed-4038-b51f-190bd94be1b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:18.425772 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44f519-f5ed-4038-b51f-190bd94be1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:18.426456 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44f519-f5ed-4038-b51f-190bd94be1b0 admin admin] PATH_INFO: `/v3/users/a6499cf76cba42aa9ffe4689cfabcf66` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:18.428214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf44f519-f5ed-4038-b51f-190bd94be1b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6499cf76cba42aa9ffe4689cfabcf66)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:18.430239 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf44f519-f5ed-4038-b51f-190bd94be1b0 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:18.439064 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f291b7a-d250-4ab9-8bab-7492d6a84a9c admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:18.445634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f56e1e-2994-4057-850b-c6b7c5955a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:18.447020 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f56e1e-2994-4057-850b-c6b7c5955a1a admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:18.447450 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f56e1e-2994-4057-850b-c6b7c5955a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:18.447869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f56e1e-2994-4057-850b-c6b7c5955a1a admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:18.448948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f56e1e-2994-4057-850b-c6b7c5955a1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:18.451359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f56e1e-2994-4057-850b-c6b7c5955a1a admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:18.464961 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1082/2145] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:47:18.469144 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f56e1e-2994-4057-850b-c6b7c5955a1a admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:18.482287 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f56e1e-2994-4057-850b-c6b7c5955a1a admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:18.488205 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27faea1d-bd9b-4fb1-9e8f-a4a24ec30ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:18.489788 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27faea1d-bd9b-4fb1-9e8f-a4a24ec30ff2 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:18.490353 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27faea1d-bd9b-4fb1-9e8f-a4a24ec30ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:18.490848 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27faea1d-bd9b-4fb1-9e8f-a4a24ec30ff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:18.491734 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27faea1d-bd9b-4fb1-9e8f-a4a24ec30ff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:18.494802 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27faea1d-bd9b-4fb1-9e8f-a4a24ec30ff2 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:18.508683 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1083/2146] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:47:18.514135 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27faea1d-bd9b-4fb1-9e8f-a4a24ec30ff2 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:18.527198 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27faea1d-bd9b-4fb1-9e8f-a4a24ec30ff2 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:18.532461 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66d54e-c3b9-4692-b467-dae5ce5e1058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:18.533626 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66d54e-c3b9-4692-b467-dae5ce5e1058 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:18.534144 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66d54e-c3b9-4692-b467-dae5ce5e1058 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:18.534890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66d54e-c3b9-4692-b467-dae5ce5e1058 admin admin] PATH_INFO: `/v3/projects/3ffa9da064c64346b5f227c28a89ea81/users/19ccda8f7ce447c490910ffa14165c8b/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:18.537756 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-bf44f519-f5ed-4038-b51f-190bd94be1b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6499cf76cba42aa9ffe4689cfabcf66'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:18.553199 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-bf44f519-f5ed-4038-b51f-190bd94be1b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6499cf76cba42aa9ffe4689cfabcf66'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:18.553958 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-bf44f519-f5ed-4038-b51f-190bd94be1b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6499cf76cba42aa9ffe4689cfabcf66'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:18.554244 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d66d54e-c3b9-4692-b467-dae5ce5e1058 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ffa9da064c64346b5f227c28a89ea81, user_id=19ccda8f7ce447c490910ffa14165c8b, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:18.557014 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d66d54e-c3b9-4692-b467-dae5ce5e1058 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:18.562634 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-bf44f519-f5ed-4038-b51f-190bd94be1b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6499cf76cba42aa9ffe4689cfabcf66'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:18.572597 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1084/2147] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:47:18 2022] PUT /identity/v3/projects/3ffa9da064c64346b5f227c28a89ea81/users/19ccda8f7ce447c490910ffa14165c8b/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:18.577305 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3e1de-57ac-4423-9d8d-8afcc5c313f2 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:18.577597 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3e1de-57ac-4423-9d8d-8afcc5c313f2 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:18.577880 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3e1de-57ac-4423-9d8d-8afcc5c313f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:18.581311 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-bf44f519-f5ed-4038-b51f-190bd94be1b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6499cf76cba42aa9ffe4689cfabcf66'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:18.592209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1064/2148] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:47:18 2022] DELETE /identity/v3/users/a6499cf76cba42aa9ffe4689cfabcf66 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:18.594247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf095da-1bfa-4205-9e9c-429cc4f07c87 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:18.594544 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf095da-1bfa-4205-9e9c-429cc4f07c87 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:18.594831 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf095da-1bfa-4205-9e9c-429cc4f07c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:18.596658 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-e7a3e1de-57ac-4423-9d8d-8afcc5c313f2 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:18.598127 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-e7a3e1de-57ac-4423-9d8d-8afcc5c313f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:18.639612 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-bbf095da-1bfa-4205-9e9c-429cc4f07c87 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:18.641660 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-bbf095da-1bfa-4205-9e9c-429cc4f07c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:18.709545 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1085/2149] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:18.720331 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3e1de-57ac-4423-9d8d-8afcc5c313f2 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:18.721824 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-e7a3e1de-57ac-4423-9d8d-8afcc5c313f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:18.765386 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1065/2150] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:47:18.816450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf095da-1bfa-4205-9e9c-429cc4f07c87 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:18.818361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-bbf095da-1bfa-4205-9e9c-429cc4f07c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:18.829678 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3e1de-57ac-4423-9d8d-8afcc5c313f2 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:18.836478 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa08b35c-4413-4082-90fe-42f41ba6d27d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:18.837450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf095da-1bfa-4205-9e9c-429cc4f07c87 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:18.837568 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08b35c-4413-4082-90fe-42f41ba6d27d service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:18.837803 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08b35c-4413-4082-90fe-42f41ba6d27d service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:18.838150 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08b35c-4413-4082-90fe-42f41ba6d27d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:18.849234 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43587f8-0ab7-46c4-b10b-98ef4ec82edd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:18.850429 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43587f8-0ab7-46c4-b10b-98ef4ec82edd service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:18.850429 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43587f8-0ab7-46c4-b10b-98ef4ec82edd service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:18.850617 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43587f8-0ab7-46c4-b10b-98ef4ec82edd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:18.851394 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa08b35c-4413-4082-90fe-42f41ba6d27d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:18.853437 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa08b35c-4413-4082-90fe-42f41ba6d27d service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:18.867408 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1086/2151] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:47:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 13:47:18.869981 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43587f8-0ab7-46c4-b10b-98ef4ec82edd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:18.871639 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43587f8-0ab7-46c4-b10b-98ef4ec82edd service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:18.883079 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1066/2152] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:47:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 13:47:19.429573 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa08b35c-4413-4082-90fe-42f41ba6d27d service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:19.431097 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-aa08b35c-4413-4082-90fe-42f41ba6d27d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:19.462224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa08b35c-4413-4082-90fe-42f41ba6d27d service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:19.468909 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568324b8-07d8-462f-88fa-c3bab1e570a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:19.469873 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568324b8-07d8-462f-88fa-c3bab1e570a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:19.470157 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568324b8-07d8-462f-88fa-c3bab1e570a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:19.470433 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568324b8-07d8-462f-88fa-c3bab1e570a2 admin admin] PATH_INFO: `/v3/projects/36d69e9074de4434af0946d86c2f809d` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:19.471875 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568324b8-07d8-462f-88fa-c3bab1e570a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36d69e9074de4434af0946d86c2f809d)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:19.474620 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568324b8-07d8-462f-88fa-c3bab1e570a2 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:19.538831 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-568324b8-07d8-462f-88fa-c3bab1e570a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36d69e9074de4434af0946d86c2f809d', 'request_id': 'req-568324b8-07d8-462f-88fa-c3bab1e570a2'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:19.541526 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:19.547785 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-568324b8-07d8-462f-88fa-c3bab1e570a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36d69e9074de4434af0946d86c2f809d', 'request_id': 'req-568324b8-07d8-462f-88fa-c3bab1e570a2'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:19.562335 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-568324b8-07d8-462f-88fa-c3bab1e570a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36d69e9074de4434af0946d86c2f809d', 'request_id': 'req-568324b8-07d8-462f-88fa-c3bab1e570a2'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:19.594016 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-568324b8-07d8-462f-88fa-c3bab1e570a2 admin admin] The token cache is being invalidate because project 36d69e9074de4434af0946d86c2f809d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:47:19.594311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-568324b8-07d8-462f-88fa-c3bab1e570a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:19.594976 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-568324b8-07d8-462f-88fa-c3bab1e570a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:19.600072 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1087/2153] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:47:19 2022] DELETE /identity/v3/projects/36d69e9074de4434af0946d86c2f809d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:19.601971 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43587f8-0ab7-46c4-b10b-98ef4ec82edd service neutron] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:19.616383 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43587f8-0ab7-46c4-b10b-98ef4ec82edd service neutron] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:19.655548 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:19.721146 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e3d173-d695-411f-9f48-4f5ea72997ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:19.722234 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3d173-d695-411f-9f48-4f5ea72997ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:19.723236 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3d173-d695-411f-9f48-4f5ea72997ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:19.723889 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3d173-d695-411f-9f48-4f5ea72997ae admin admin] PATH_INFO: `/v3/users/b0116bcfeee144e0908e93b9a7661613` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:19.734987 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e3d173-d695-411f-9f48-4f5ea72997ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0116bcfeee144e0908e93b9a7661613)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:19.737453 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e3d173-d695-411f-9f48-4f5ea72997ae admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:19.846193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-e8e3d173-d695-411f-9f48-4f5ea72997ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0116bcfeee144e0908e93b9a7661613'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:19.857665 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-e8e3d173-d695-411f-9f48-4f5ea72997ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0116bcfeee144e0908e93b9a7661613'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:19.858658 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-e8e3d173-d695-411f-9f48-4f5ea72997ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0116bcfeee144e0908e93b9a7661613'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:19.867216 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-e8e3d173-d695-411f-9f48-4f5ea72997ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0116bcfeee144e0908e93b9a7661613'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:19.882032 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-e8e3d173-d695-411f-9f48-4f5ea72997ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0116bcfeee144e0908e93b9a7661613'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:19.889633 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1067/2154] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:47:19 2022] DELETE /identity/v3/users/b0116bcfeee144e0908e93b9a7661613 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:20.799069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568324b8-07d8-462f-88fa-c3bab1e570a2 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:20.804504 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-568324b8-07d8-462f-88fa-c3bab1e570a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:20.839015 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568324b8-07d8-462f-88fa-c3bab1e570a2 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:20.928236 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f3a7bb-0966-4e1b-9958-7942c660d1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:20.929508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3a7bb-0966-4e1b-9958-7942c660d1b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:20.930521 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3a7bb-0966-4e1b-9958-7942c660d1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:20.930807 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3a7bb-0966-4e1b-9958-7942c660d1b5 admin admin] PATH_INFO: `/v3/projects/9d6cc98e9115430f94da07fc567a48e7` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:20.932874 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f3a7bb-0966-4e1b-9958-7942c660d1b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d6cc98e9115430f94da07fc567a48e7)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:20.935333 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f3a7bb-0966-4e1b-9958-7942c660d1b5 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:21.004279 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-d9f3a7bb-0966-4e1b-9958-7942c660d1b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d6cc98e9115430f94da07fc567a48e7', 'request_id': 'req-d9f3a7bb-0966-4e1b-9958-7942c660d1b5'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:21.012104 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-d9f3a7bb-0966-4e1b-9958-7942c660d1b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d6cc98e9115430f94da07fc567a48e7', 'request_id': 'req-d9f3a7bb-0966-4e1b-9958-7942c660d1b5'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:21.055237 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-d9f3a7bb-0966-4e1b-9958-7942c660d1b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d6cc98e9115430f94da07fc567a48e7', 'request_id': 'req-d9f3a7bb-0966-4e1b-9958-7942c660d1b5'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:21.071977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-d9f3a7bb-0966-4e1b-9958-7942c660d1b5 admin admin] The token cache is being invalidate because project 9d6cc98e9115430f94da07fc567a48e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:47:21.072281 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-d9f3a7bb-0966-4e1b-9958-7942c660d1b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:21.072957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-d9f3a7bb-0966-4e1b-9958-7942c660d1b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:21.074785 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1088/2155] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:47:20 2022] DELETE /identity/v3/projects/9d6cc98e9115430f94da07fc567a48e7 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:21.079476 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15868273-04dc-48b9-ae81-859c55714690 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:21.079933 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15868273-04dc-48b9-ae81-859c55714690 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:21.080060 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15868273-04dc-48b9-ae81-859c55714690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:21.105333 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-15868273-04dc-48b9-ae81-859c55714690 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:21.107678 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-15868273-04dc-48b9-ae81-859c55714690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:21.211580 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1068/2156] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:21.220877 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25263735-66b3-488e-aafb-2a83f644127f None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:21.221320 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25263735-66b3-488e-aafb-2a83f644127f None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:21.221641 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25263735-66b3-488e-aafb-2a83f644127f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:21.250360 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-25263735-66b3-488e-aafb-2a83f644127f None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:21.251921 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-25263735-66b3-488e-aafb-2a83f644127f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:21.267622 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1089/2157] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:21.272558 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15868273-04dc-48b9-ae81-859c55714690 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:21.284842 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15868273-04dc-48b9-ae81-859c55714690 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:21.290824 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2439dea0-e5fe-4d00-96dc-f38cbb46791a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:21.291511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439dea0-e5fe-4d00-96dc-f38cbb46791a admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:21.292050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439dea0-e5fe-4d00-96dc-f38cbb46791a admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:21.292333 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439dea0-e5fe-4d00-96dc-f38cbb46791a admin admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:21.293117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2439dea0-e5fe-4d00-96dc-f38cbb46791a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:47:21.294247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2439dea0-e5fe-4d00-96dc-f38cbb46791a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:21.295563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2439dea0-e5fe-4d00-96dc-f38cbb46791a admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:21.321841 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1069/2158] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:47:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:47:21.326984 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25263735-66b3-488e-aafb-2a83f644127f None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:21.338101 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25263735-66b3-488e-aafb-2a83f644127f None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:21.352433 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b82cbe-371c-4e49-b9b3-f468206ad865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:21.353365 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b82cbe-371c-4e49-b9b3-f468206ad865 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:21.353481 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b82cbe-371c-4e49-b9b3-f468206ad865 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:21.353703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b82cbe-371c-4e49-b9b3-f468206ad865 admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:21.354423 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b82cbe-371c-4e49-b9b3-f468206ad865 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:21.356537 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b82cbe-371c-4e49-b9b3-f468206ad865 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:21.450511 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1090/2159] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:47:21 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:21.455526 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2439dea0-e5fe-4d00-96dc-f38cbb46791a admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:21.468797 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2439dea0-e5fe-4d00-96dc-f38cbb46791a admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:21.527269 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:21.568929 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c2009-f26d-416a-8884-7ad6a585720e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:21.569943 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c2009-f26d-416a-8884-7ad6a585720e admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:21.571025 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c2009-f26d-416a-8884-7ad6a585720e admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:21.571313 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c2009-f26d-416a-8884-7ad6a585720e admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:21.572022 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c2009-f26d-416a-8884-7ad6a585720e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:21.574037 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c2009-f26d-416a-8884-7ad6a585720e admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:21.611674 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1070/2160] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:47:21 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:21.612919 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b82cbe-371c-4e49-b9b3-f468206ad865 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:21.624653 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b82cbe-371c-4e49-b9b3-f468206ad865 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:21.633769 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3727a8-a244-4a29-851b-5a9212e51e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:21.635014 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3727a8-a244-4a29-851b-5a9212e51e84 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:21.636355 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3727a8-a244-4a29-851b-5a9212e51e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:21.636825 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3727a8-a244-4a29-851b-5a9212e51e84 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:21.637826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3727a8-a244-4a29-851b-5a9212e51e84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:21.641422 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3727a8-a244-4a29-851b-5a9212e51e84 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:21.663401 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1091/2161] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:47:21.667294 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c2009-f26d-416a-8884-7ad6a585720e admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:21.678337 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c2009-f26d-416a-8884-7ad6a585720e admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:21.683385 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ff7e8-116f-4a5c-adac-4f345193e778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:21.684314 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ff7e8-116f-4a5c-adac-4f345193e778 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:21.684760 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ff7e8-116f-4a5c-adac-4f345193e778 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:21.685068 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ff7e8-116f-4a5c-adac-4f345193e778 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:21.685847 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ff7e8-116f-4a5c-adac-4f345193e778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:21.687672 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ff7e8-116f-4a5c-adac-4f345193e778 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:21.699646 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1071/2162] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:47:21.703925 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3727a8-a244-4a29-851b-5a9212e51e84 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:21.726560 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3727a8-a244-4a29-851b-5a9212e51e84 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:21.733141 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442fa95-9439-45cf-91a4-92c4e4555857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:21.734082 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442fa95-9439-45cf-91a4-92c4e4555857 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:21.734454 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442fa95-9439-45cf-91a4-92c4e4555857 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:21.735818 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442fa95-9439-45cf-91a4-92c4e4555857 admin admin] PATH_INFO: `/v3/projects/a9a20fc57e104d31b0b0c93fce8ccb32/users/3495c193a38b47789324dd35acde594f/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:21.756614 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c442fa95-9439-45cf-91a4-92c4e4555857 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9a20fc57e104d31b0b0c93fce8ccb32, user_id=3495c193a38b47789324dd35acde594f, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:21.759841 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c442fa95-9439-45cf-91a4-92c4e4555857 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:21.781261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1092/2163] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:47:21 2022] PUT /identity/v3/projects/a9a20fc57e104d31b0b0c93fce8ccb32/users/3495c193a38b47789324dd35acde594f/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:21.787272 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d6b7f-010c-414f-8cb2-9901fd227e3a None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:21.787590 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d6b7f-010c-414f-8cb2-9901fd227e3a None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:21.787868 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d6b7f-010c-414f-8cb2-9901fd227e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:21.823583 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-e51d6b7f-010c-414f-8cb2-9901fd227e3a None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:21.827730 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-e51d6b7f-010c-414f-8cb2-9901fd227e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:21.939862 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1072/2164] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:21.981585 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442fa95-9439-45cf-91a4-92c4e4555857 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:21.984214 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-c442fa95-9439-45cf-91a4-92c4e4555857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:21.996608 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:22.076351 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442fa95-9439-45cf-91a4-92c4e4555857 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:22.080514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e81fae-9242-4035-a3f6-9a78f2fc286c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:22.081369 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e81fae-9242-4035-a3f6-9a78f2fc286c service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:22.081509 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e81fae-9242-4035-a3f6-9a78f2fc286c service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:22.081744 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e81fae-9242-4035-a3f6-9a78f2fc286c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:22.093027 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e81fae-9242-4035-a3f6-9a78f2fc286c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:22.094527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e81fae-9242-4035-a3f6-9a78f2fc286c service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:22.117574 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1093/2165] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:47:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:47:22.608520 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa187ab-ad60-4892-9479-f36d651de863 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:22.609057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa187ab-ad60-4892-9479-f36d651de863 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:22.609153 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa187ab-ad60-4892-9479-f36d651de863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:22.628096 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-8fa187ab-ad60-4892-9479-f36d651de863 None None] MFA Rules not processed for user `050d5eb22b2c45ee85eb8c118b9ef21e`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:22.629905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-8fa187ab-ad60-4892-9479-f36d651de863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:22.702590 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1073/2166] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:25.247502 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6903e-cdd3-4f89-afed-f7f46add82c7 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:25.248047 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6903e-cdd3-4f89-afed-f7f46add82c7 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:25.248047 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6903e-cdd3-4f89-afed-f7f46add82c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:25.305349 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-a4c6903e-cdd3-4f89-afed-f7f46add82c7 None None] MFA Rules not processed for user `3495c193a38b47789324dd35acde594f`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:25.306782 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-a4c6903e-cdd3-4f89-afed-f7f46add82c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:25.531117 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1094/2167] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:25.536370 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa187ab-ad60-4892-9479-f36d651de863 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:25.559446 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa187ab-ad60-4892-9479-f36d651de863 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:25.576740 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df664d-1191-4cf8-8557-64fe5bb18cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:25.577789 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df664d-1191-4cf8-8557-64fe5bb18cc7 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:25.578218 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df664d-1191-4cf8-8557-64fe5bb18cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:25.578628 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df664d-1191-4cf8-8557-64fe5bb18cc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:25.579689 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13df664d-1191-4cf8-8557-64fe5bb18cc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:25.583362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13df664d-1191-4cf8-8557-64fe5bb18cc7 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:25.673554 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1074/2168] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:47:25 2022] POST /identity/v3/projects => generated 371 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:25.678394 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6903e-cdd3-4f89-afed-f7f46add82c7 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:25.717485 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6903e-cdd3-4f89-afed-f7f46add82c7 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:25.876110 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8b5c3-641a-4a30-9ba8-e05e9d8704d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:25.876983 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8b5c3-641a-4a30-9ba8-e05e9d8704d1 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:25.877489 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8b5c3-641a-4a30-9ba8-e05e9d8704d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:25.877863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8b5c3-641a-4a30-9ba8-e05e9d8704d1 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:25.878657 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e8b5c3-641a-4a30-9ba8-e05e9d8704d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:25.881834 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e8b5c3-641a-4a30-9ba8-e05e9d8704d1 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:25.906775 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1095/2169] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:47:25 2022] POST /identity/v3/users => generated 292 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:25.911418 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df664d-1191-4cf8-8557-64fe5bb18cc7 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:25.934147 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df664d-1191-4cf8-8557-64fe5bb18cc7 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:25.981924 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e5eed-7ba8-4658-8c74-93eb11371dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:25.982947 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e5eed-7ba8-4658-8c74-93eb11371dc3 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:25.984513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e5eed-7ba8-4658-8c74-93eb11371dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:25.984968 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e5eed-7ba8-4658-8c74-93eb11371dc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:25.987125 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e5eed-7ba8-4658-8c74-93eb11371dc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:25.990208 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e5eed-7ba8-4658-8c74-93eb11371dc3 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:26.009582 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1075/2170] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:47:26.014727 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8b5c3-641a-4a30-9ba8-e05e9d8704d1 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:26.027559 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8b5c3-641a-4a30-9ba8-e05e9d8704d1 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:26.033334 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6981147-8b04-4f4b-96e7-1eaa16cd2a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:26.035115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6981147-8b04-4f4b-96e7-1eaa16cd2a14 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:26.035955 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6981147-8b04-4f4b-96e7-1eaa16cd2a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:26.036429 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6981147-8b04-4f4b-96e7-1eaa16cd2a14 admin admin] PATH_INFO: `/v3/projects/99d3842d090b45ce98a44f21aac8b464/users/54065e9b627f4f26b120f1cc7fd169b5/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:26.056957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6981147-8b04-4f4b-96e7-1eaa16cd2a14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99d3842d090b45ce98a44f21aac8b464, user_id=54065e9b627f4f26b120f1cc7fd169b5, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:26.059953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6981147-8b04-4f4b-96e7-1eaa16cd2a14 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:26.077831 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1096/2171] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:47:26 2022] PUT /identity/v3/projects/99d3842d090b45ce98a44f21aac8b464/users/54065e9b627f4f26b120f1cc7fd169b5/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:26.082566 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e5eed-7ba8-4658-8c74-93eb11371dc3 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:26.096898 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e5eed-7ba8-4658-8c74-93eb11371dc3 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:26.120203 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:26.168908 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b44c0-bc32-43e5-8e76-f0a0c45a72e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:26.169714 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b44c0-bc32-43e5-8e76-f0a0c45a72e2 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:26.170493 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b44c0-bc32-43e5-8e76-f0a0c45a72e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:26.170493 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b44c0-bc32-43e5-8e76-f0a0c45a72e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:26.171284 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b44c0-bc32-43e5-8e76-f0a0c45a72e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:26.172991 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b44c0-bc32-43e5-8e76-f0a0c45a72e2 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:26.185522 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1076/2172] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:26 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:47:26.188418 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6981147-8b04-4f4b-96e7-1eaa16cd2a14 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:26.195859 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:26.200217 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6981147-8b04-4f4b-96e7-1eaa16cd2a14 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:26.213430 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e226e60-a182-4847-97f7-b83e701891e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:26.214928 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e226e60-a182-4847-97f7-b83e701891e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:26.215371 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e226e60-a182-4847-97f7-b83e701891e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:26.216522 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e226e60-a182-4847-97f7-b83e701891e1 admin admin] PATH_INFO: `/v3/projects/99d3842d090b45ce98a44f21aac8b464/users/54065e9b627f4f26b120f1cc7fd169b5/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:26.219092 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e226e60-a182-4847-97f7-b83e701891e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99d3842d090b45ce98a44f21aac8b464, user_id=54065e9b627f4f26b120f1cc7fd169b5, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:26.222161 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e226e60-a182-4847-97f7-b83e701891e1 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:26.238102 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1097/2173] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:47:26 2022] PUT /identity/v3/projects/99d3842d090b45ce98a44f21aac8b464/users/54065e9b627f4f26b120f1cc7fd169b5/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:26.943008 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fe361-4afa-4951-8a59-44de0f68e8af None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:26.943502 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fe361-4afa-4951-8a59-44de0f68e8af None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:26.943859 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fe361-4afa-4951-8a59-44de0f68e8af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:26.994225 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-f69fe361-4afa-4951-8a59-44de0f68e8af None None] MFA Rules not processed for user `19ccda8f7ce447c490910ffa14165c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:26.996217 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-f69fe361-4afa-4951-8a59-44de0f68e8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:27.152173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1077/2174] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:47:27.156718 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e226e60-a182-4847-97f7-b83e701891e1 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:27.159104 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-0e226e60-a182-4847-97f7-b83e701891e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:27.245389 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e226e60-a182-4847-97f7-b83e701891e1 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:27.251802 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44f06b-2485-4588-a859-d7c52d54a7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:27.252739 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44f06b-2485-4588-a859-d7c52d54a7c6 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:27.253253 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44f06b-2485-4588-a859-d7c52d54a7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:27.254195 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44f06b-2485-4588-a859-d7c52d54a7c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:27.255192 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44f06b-2485-4588-a859-d7c52d54a7c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:27.257333 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44f06b-2485-4588-a859-d7c52d54a7c6 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:27.287921 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1098/2175] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:47:27 2022] POST /identity/v3/projects => generated 383 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:27.292614 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69fe361-4afa-4951-8a59-44de0f68e8af None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:27.327017 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69fe361-4afa-4951-8a59-44de0f68e8af None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:27.402574 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f479e16e-889f-475a-9982-425074ff84d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:27.403271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f479e16e-889f-475a-9982-425074ff84d1 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:27.404066 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f479e16e-889f-475a-9982-425074ff84d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:27.404522 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f479e16e-889f-475a-9982-425074ff84d1 admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:27.405472 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f479e16e-889f-475a-9982-425074ff84d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:27.409331 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f479e16e-889f-475a-9982-425074ff84d1 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:27.443095 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1078/2176] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:47:27 2022] POST /identity/v3/users => generated 298 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:47:27.446888 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44f06b-2485-4588-a859-d7c52d54a7c6 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:27.469849 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44f06b-2485-4588-a859-d7c52d54a7c6 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:27.482759 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b3924-0726-4660-a75d-8174c5979b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:27.484220 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b3924-0726-4660-a75d-8174c5979b99 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:27.484663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b3924-0726-4660-a75d-8174c5979b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:27.485247 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b3924-0726-4660-a75d-8174c5979b99 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:27.486969 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b3924-0726-4660-a75d-8174c5979b99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:27.491842 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b3924-0726-4660-a75d-8174c5979b99 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:27.532609 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1099/2177] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:27 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:47:27.537600 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f479e16e-889f-475a-9982-425074ff84d1 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:27.550344 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f479e16e-889f-475a-9982-425074ff84d1 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:27.556994 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7668664a-fcdf-46aa-b4cd-96589ccac6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:27.558129 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668664a-fcdf-46aa-b4cd-96589ccac6f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:27.558934 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668664a-fcdf-46aa-b4cd-96589ccac6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:27.559586 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668664a-fcdf-46aa-b4cd-96589ccac6f2 admin admin] PATH_INFO: `/v3/projects/990f781d3ce943d2a69b9ca6ce3ea99c/users/7dbe70ba377644ba9c146a092655b02e/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:27.585957 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7668664a-fcdf-46aa-b4cd-96589ccac6f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=990f781d3ce943d2a69b9ca6ce3ea99c, user_id=7dbe70ba377644ba9c146a092655b02e, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:27.591580 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7668664a-fcdf-46aa-b4cd-96589ccac6f2 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:27.618755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1079/2178] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:47:27 2022] PUT /identity/v3/projects/990f781d3ce943d2a69b9ca6ce3ea99c/users/7dbe70ba377644ba9c146a092655b02e/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:47:27.622925 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b3924-0726-4660-a75d-8174c5979b99 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:27.655455 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b3924-0726-4660-a75d-8174c5979b99 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:27.783434 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eee7d2-8668-482e-b568-f21372f721cc None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:27.783966 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eee7d2-8668-482e-b568-f21372f721cc None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:27.784394 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eee7d2-8668-482e-b568-f21372f721cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:27.808609 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e00dc-0a39-4fcf-9665-9c41ae9cf05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:27.809232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e00dc-0a39-4fcf-9665-9c41ae9cf05d admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:27.809519 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e00dc-0a39-4fcf-9665-9c41ae9cf05d admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:27.809799 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e00dc-0a39-4fcf-9665-9c41ae9cf05d admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:27.810571 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e00dc-0a39-4fcf-9665-9c41ae9cf05d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:27.812747 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e00dc-0a39-4fcf-9665-9c41ae9cf05d admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:27.828401 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1100/2179] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:27 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:47:27.829949 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e00dc-0a39-4fcf-9665-9c41ae9cf05d admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:27.845787 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e00dc-0a39-4fcf-9665-9c41ae9cf05d admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:27.847023 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-26eee7d2-8668-482e-b568-f21372f721cc None None] MFA Rules not processed for user `54065e9b627f4f26b120f1cc7fd169b5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:27.849391 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-26eee7d2-8668-482e-b568-f21372f721cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:27.853746 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7c95e-f00b-4608-a0ae-332b3c74e0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:27.854847 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7c95e-f00b-4608-a0ae-332b3c74e0fe admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:27.855138 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7c95e-f00b-4608-a0ae-332b3c74e0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:27.855464 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7c95e-f00b-4608-a0ae-332b3c74e0fe admin admin] PATH_INFO: `/v3/projects/990f781d3ce943d2a69b9ca6ce3ea99c/users/7dbe70ba377644ba9c146a092655b02e/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:27.860645 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a7c95e-f00b-4608-a0ae-332b3c74e0fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=990f781d3ce943d2a69b9ca6ce3ea99c, user_id=7dbe70ba377644ba9c146a092655b02e, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:27.863903 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a7c95e-f00b-4608-a0ae-332b3c74e0fe admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:27.864692 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:27.886229 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1101/2180] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:47:27 2022] PUT /identity/v3/projects/990f781d3ce943d2a69b9ca6ce3ea99c/users/7dbe70ba377644ba9c146a092655b02e/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:27.962520 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1080/2181] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:27 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:27.974941 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7c95e-f00b-4608-a0ae-332b3c74e0fe admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:27.978093 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-61a7c95e-f00b-4608-a0ae-332b3c74e0fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:28.105973 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:28.110985 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7c95e-f00b-4608-a0ae-332b3c74e0fe admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:28.118362 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a2c12-b83b-46a2-8ab2-321602c95958 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:28.138961 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a2c12-b83b-46a2-8ab2-321602c95958 service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:28.139289 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a2c12-b83b-46a2-8ab2-321602c95958 service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:28.139604 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a2c12-b83b-46a2-8ab2-321602c95958 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:28.171655 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eee7d2-8668-482e-b568-f21372f721cc None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:28.173338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-26eee7d2-8668-482e-b568-f21372f721cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:28.197703 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eee7d2-8668-482e-b568-f21372f721cc None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:28.360728 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0515e1c7-aea5-447c-a4e5-b6759d56e5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:28.367402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515e1c7-aea5-447c-a4e5-b6759d56e5d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:28.367939 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515e1c7-aea5-447c-a4e5-b6759d56e5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:28.369297 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515e1c7-aea5-447c-a4e5-b6759d56e5d2 admin admin] PATH_INFO: `/v3/users/ad0781ce20e54c3a9b7c61e0cccf6918` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:28.374024 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0515e1c7-aea5-447c-a4e5-b6759d56e5d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad0781ce20e54c3a9b7c61e0cccf6918)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:28.375423 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3a2c12-b83b-46a2-8ab2-321602c95958 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:28.378515 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3a2c12-b83b-46a2-8ab2-321602c95958 service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:28.383985 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0515e1c7-aea5-447c-a4e5-b6759d56e5d2 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:28.404962 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1102/2182] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:47:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 13:47:28.539162 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-0515e1c7-aea5-447c-a4e5-b6759d56e5d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad0781ce20e54c3a9b7c61e0cccf6918'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:28.556156 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-0515e1c7-aea5-447c-a4e5-b6759d56e5d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad0781ce20e54c3a9b7c61e0cccf6918'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:28.557602 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-0515e1c7-aea5-447c-a4e5-b6759d56e5d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad0781ce20e54c3a9b7c61e0cccf6918'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:28.574355 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-0515e1c7-aea5-447c-a4e5-b6759d56e5d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad0781ce20e54c3a9b7c61e0cccf6918'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:28.602215 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-0515e1c7-aea5-447c-a4e5-b6759d56e5d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad0781ce20e54c3a9b7c61e0cccf6918'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:28.616604 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1081/2183] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:47:28 2022] DELETE /identity/v3/users/ad0781ce20e54c3a9b7c61e0cccf6918 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:28.999581 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a2c12-b83b-46a2-8ab2-321602c95958 service cinder] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:29.022412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a2c12-b83b-46a2-8ab2-321602c95958 service cinder] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:29.159785 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678c393-7351-4a39-adfe-372428c89f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:29.161398 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678c393-7351-4a39-adfe-372428c89f41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:29.161792 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678c393-7351-4a39-adfe-372428c89f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:29.162169 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678c393-7351-4a39-adfe-372428c89f41 admin admin] PATH_INFO: `/v3/projects/d48e24dbeaa241d7a97350fb2c8187b9` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:29.167468 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2678c393-7351-4a39-adfe-372428c89f41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d48e24dbeaa241d7a97350fb2c8187b9)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:29.173097 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2678c393-7351-4a39-adfe-372428c89f41 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:29.334974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-2678c393-7351-4a39-adfe-372428c89f41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd48e24dbeaa241d7a97350fb2c8187b9', 'request_id': 'req-2678c393-7351-4a39-adfe-372428c89f41'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:29.342953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-2678c393-7351-4a39-adfe-372428c89f41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd48e24dbeaa241d7a97350fb2c8187b9', 'request_id': 'req-2678c393-7351-4a39-adfe-372428c89f41'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:29.359569 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-2678c393-7351-4a39-adfe-372428c89f41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd48e24dbeaa241d7a97350fb2c8187b9', 'request_id': 'req-2678c393-7351-4a39-adfe-372428c89f41'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:29.376269 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-2678c393-7351-4a39-adfe-372428c89f41 admin admin] The token cache is being invalidate because project d48e24dbeaa241d7a97350fb2c8187b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:47:29.376918 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-2678c393-7351-4a39-adfe-372428c89f41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:29.380345 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-2678c393-7351-4a39-adfe-372428c89f41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:29.382660 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1103/2184] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:47:28 2022] DELETE /identity/v3/projects/d48e24dbeaa241d7a97350fb2c8187b9 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:29.387875 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a196c-b5aa-44ee-8c8a-396192ddc350 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:29.388707 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a196c-b5aa-44ee-8c8a-396192ddc350 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:29.389194 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a196c-b5aa-44ee-8c8a-396192ddc350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:29.413078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-334a196c-b5aa-44ee-8c8a-396192ddc350 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:29.415921 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-334a196c-b5aa-44ee-8c8a-396192ddc350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:29.432967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:29.511504 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f760815-42df-414b-9fb4-21fb9c8f06d3 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:29.512045 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f760815-42df-414b-9fb4-21fb9c8f06d3 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:29.512470 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f760815-42df-414b-9fb4-21fb9c8f06d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:29.513492 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1082/2185] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:29.537371 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-5f760815-42df-414b-9fb4-21fb9c8f06d3 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:29.538876 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-5f760815-42df-414b-9fb4-21fb9c8f06d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:29.552172 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1104/2186] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:47:29.553135 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334a196c-b5aa-44ee-8c8a-396192ddc350 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:29.579254 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334a196c-b5aa-44ee-8c8a-396192ddc350 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:29.594642 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd83226-2dd5-447f-b692-42a0f71cfa8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:29.595559 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd83226-2dd5-447f-b692-42a0f71cfa8a admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:29.597429 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd83226-2dd5-447f-b692-42a0f71cfa8a admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:29.597907 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd83226-2dd5-447f-b692-42a0f71cfa8a admin admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:29.599102 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd83226-2dd5-447f-b692-42a0f71cfa8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:47:29.600983 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd83226-2dd5-447f-b692-42a0f71cfa8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:29.602652 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd83226-2dd5-447f-b692-42a0f71cfa8a admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:29.636712 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f760815-42df-414b-9fb4-21fb9c8f06d3 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:29.646283 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1083/2187] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:47:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:47:29.651969 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f760815-42df-414b-9fb4-21fb9c8f06d3 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:29.657473 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6edd52a-5ff1-4681-973e-4d12ac9ba00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:29.658348 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edd52a-5ff1-4681-973e-4d12ac9ba00e admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:29.658750 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edd52a-5ff1-4681-973e-4d12ac9ba00e admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:29.659259 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edd52a-5ff1-4681-973e-4d12ac9ba00e admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:29.660183 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6edd52a-5ff1-4681-973e-4d12ac9ba00e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:29.662393 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6edd52a-5ff1-4681-973e-4d12ac9ba00e admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:29.711817 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1105/2188] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:47:29 2022] POST /identity/v3/projects => generated 355 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:29.716705 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd83226-2dd5-447f-b692-42a0f71cfa8a admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:29.742082 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd83226-2dd5-447f-b692-42a0f71cfa8a admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:29.870379 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640754f0-472a-4c81-9e4b-2735ffca4f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:29.871415 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640754f0-472a-4c81-9e4b-2735ffca4f9c admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:29.871909 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640754f0-472a-4c81-9e4b-2735ffca4f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:29.872403 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640754f0-472a-4c81-9e4b-2735ffca4f9c admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:29.873353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640754f0-472a-4c81-9e4b-2735ffca4f9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:29.876079 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640754f0-472a-4c81-9e4b-2735ffca4f9c admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:29.905887 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1084/2189] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:47:29 2022] POST /identity/v3/users => generated 278 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:29.911042 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6edd52a-5ff1-4681-973e-4d12ac9ba00e admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:29.927900 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:29.934641 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6edd52a-5ff1-4681-973e-4d12ac9ba00e admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:29.945558 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae62e2a-5551-4fcf-a179-05d34940decc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:29.946769 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae62e2a-5551-4fcf-a179-05d34940decc admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:29.948138 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae62e2a-5551-4fcf-a179-05d34940decc admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:29.948506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae62e2a-5551-4fcf-a179-05d34940decc admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:29.950191 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae62e2a-5551-4fcf-a179-05d34940decc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:29.952536 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae62e2a-5551-4fcf-a179-05d34940decc admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:29.968514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1106/2190] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:29 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:47:29.972396 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640754f0-472a-4c81-9e4b-2735ffca4f9c admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:29.988721 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640754f0-472a-4c81-9e4b-2735ffca4f9c admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:30.001114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4f0eb-dedf-401f-b55c-a976be86b761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:30.001449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4f0eb-dedf-401f-b55c-a976be86b761 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:30.001686 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4f0eb-dedf-401f-b55c-a976be86b761 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:30.001963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4f0eb-dedf-401f-b55c-a976be86b761 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:30.002707 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f4f0eb-dedf-401f-b55c-a976be86b761 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:30.004415 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f4f0eb-dedf-401f-b55c-a976be86b761 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:30.026894 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1085/2191] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:29 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:47:30.030428 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae62e2a-5551-4fcf-a179-05d34940decc admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:30.064663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae62e2a-5551-4fcf-a179-05d34940decc admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:30.083025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122e1e3-f0d7-466a-9e17-444973445080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:30.095834 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122e1e3-f0d7-466a-9e17-444973445080 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:30.096700 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122e1e3-f0d7-466a-9e17-444973445080 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:30.097029 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122e1e3-f0d7-466a-9e17-444973445080 admin admin] PATH_INFO: `/v3/projects/93feebcc3429463aaa122941fa66f29c/users/f18b2eb526ed4085a6ea694198b56d9b/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:30.152591 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122e1e3-f0d7-466a-9e17-444973445080 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93feebcc3429463aaa122941fa66f29c, user_id=f18b2eb526ed4085a6ea694198b56d9b, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:30.158480 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122e1e3-f0d7-466a-9e17-444973445080 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:30.196649 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1107/2192] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:47:30 2022] PUT /identity/v3/projects/93feebcc3429463aaa122941fa66f29c/users/f18b2eb526ed4085a6ea694198b56d9b/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:30.208267 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d93d4-b796-4dc7-8e1f-4d598baf4973 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:30.208853 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d93d4-b796-4dc7-8e1f-4d598baf4973 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:30.209439 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d93d4-b796-4dc7-8e1f-4d598baf4973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:30.251737 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122e1e3-f0d7-466a-9e17-444973445080 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:30.254535 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-6122e1e3-f0d7-466a-9e17-444973445080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:30.282852 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-c65d93d4-b796-4dc7-8e1f-4d598baf4973 None None] MFA Rules not processed for user `f18b2eb526ed4085a6ea694198b56d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:30.284251 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-c65d93d4-b796-4dc7-8e1f-4d598baf4973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:30.432223 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122e1e3-f0d7-466a-9e17-444973445080 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:30.445253 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8a802-3c8c-4f7a-9e5c-37b010570526 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:30.446799 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8a802-3c8c-4f7a-9e5c-37b010570526 service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:30.448026 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8a802-3c8c-4f7a-9e5c-37b010570526 service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:30.448586 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8a802-3c8c-4f7a-9e5c-37b010570526 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:30.455720 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-86b8a802-3c8c-4f7a-9e5c-37b010570526 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:30.484525 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1086/2193] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:30 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:30.486443 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d93d4-b796-4dc7-8e1f-4d598baf4973 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:30.553794 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:30.647584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b8a802-3c8c-4f7a-9e5c-37b010570526 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:30.649183 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b8a802-3c8c-4f7a-9e5c-37b010570526 service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:30.689136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d93d4-b796-4dc7-8e1f-4d598baf4973 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:30.695456 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1108/2194] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:47:30 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 13:47:30.700686 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff83afd6-2dd7-46c5-963e-4e2cf450d7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:30.701511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83afd6-2dd7-46c5-963e-4e2cf450d7ed admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:30.701807 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83afd6-2dd7-46c5-963e-4e2cf450d7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:30.702076 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83afd6-2dd7-46c5-963e-4e2cf450d7ed admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:30.702782 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff83afd6-2dd7-46c5-963e-4e2cf450d7ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:30.710040 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff83afd6-2dd7-46c5-963e-4e2cf450d7ed admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:30.766146 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1087/2195] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:47:30 2022] POST /identity/v3/projects => generated 355 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:30.767342 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8a802-3c8c-4f7a-9e5c-37b010570526 service cinder] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:30.786591 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8a802-3c8c-4f7a-9e5c-37b010570526 service cinder] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:31.000878 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c10375-fdcc-498a-926c-566e78422a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:31.001642 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c10375-fdcc-498a-926c-566e78422a1e admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:31.005243 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c10375-fdcc-498a-926c-566e78422a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:31.006043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c10375-fdcc-498a-926c-566e78422a1e admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:31.008875 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c10375-fdcc-498a-926c-566e78422a1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:31.018247 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c10375-fdcc-498a-926c-566e78422a1e admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:31.139024 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1109/2196] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:47:30 2022] POST /identity/v3/users => generated 284 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:31.139435 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff83afd6-2dd7-46c5-963e-4e2cf450d7ed admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:31.163928 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff83afd6-2dd7-46c5-963e-4e2cf450d7ed admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:31.186640 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c74045-cf91-474a-96ef-83b315440b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:31.187410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c74045-cf91-474a-96ef-83b315440b59 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:31.187787 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c74045-cf91-474a-96ef-83b315440b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:31.187966 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c74045-cf91-474a-96ef-83b315440b59 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:31.188757 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c74045-cf91-474a-96ef-83b315440b59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:31.190570 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c74045-cf91-474a-96ef-83b315440b59 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:31.207438 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1088/2197] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:31 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:47:31.209067 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c10375-fdcc-498a-926c-566e78422a1e admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:31.226652 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c10375-fdcc-498a-926c-566e78422a1e admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:31.234376 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ca5f9-a081-44e6-8eee-1224b13d47a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:31.235961 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ca5f9-a081-44e6-8eee-1224b13d47a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:31.236326 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ca5f9-a081-44e6-8eee-1224b13d47a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:31.236715 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ca5f9-a081-44e6-8eee-1224b13d47a1 admin admin] PATH_INFO: `/v3/projects/c1254f0335304a4fa2a9aca11fe59dc5/users/e8ecdb56dd02499ab0d0cd1e0bc31fea/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:31.242619 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:31.282839 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ca5f9-a081-44e6-8eee-1224b13d47a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1254f0335304a4fa2a9aca11fe59dc5, user_id=e8ecdb56dd02499ab0d0cd1e0bc31fea, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:31.290587 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ca5f9-a081-44e6-8eee-1224b13d47a1 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:31.334921 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1110/2198] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:47:31 2022] PUT /identity/v3/projects/c1254f0335304a4fa2a9aca11fe59dc5/users/e8ecdb56dd02499ab0d0cd1e0bc31fea/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:31.335209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c74045-cf91-474a-96ef-83b315440b59 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:31.350437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c74045-cf91-474a-96ef-83b315440b59 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:31.487671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589728f-1c23-44f8-baf9-50f13ad4ff77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:31.488563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589728f-1c23-44f8-baf9-50f13ad4ff77 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:31.492565 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589728f-1c23-44f8-baf9-50f13ad4ff77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:31.492565 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589728f-1c23-44f8-baf9-50f13ad4ff77 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:31.498804 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2589728f-1c23-44f8-baf9-50f13ad4ff77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:31.500679 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2589728f-1c23-44f8-baf9-50f13ad4ff77 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:31.517155 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1089/2199] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:31 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:47:31.522245 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ca5f9-a081-44e6-8eee-1224b13d47a1 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:31.537398 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ca5f9-a081-44e6-8eee-1224b13d47a1 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:31.548071 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efba2f3-dc24-4ba0-bf72-6dab465e3206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:31.549661 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efba2f3-dc24-4ba0-bf72-6dab465e3206 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:31.550146 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efba2f3-dc24-4ba0-bf72-6dab465e3206 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:31.551880 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efba2f3-dc24-4ba0-bf72-6dab465e3206 admin admin] PATH_INFO: `/v3/projects/c1254f0335304a4fa2a9aca11fe59dc5/users/e8ecdb56dd02499ab0d0cd1e0bc31fea/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:31.556833 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efba2f3-dc24-4ba0-bf72-6dab465e3206 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1254f0335304a4fa2a9aca11fe59dc5, user_id=e8ecdb56dd02499ab0d0cd1e0bc31fea, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:31.564268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efba2f3-dc24-4ba0-bf72-6dab465e3206 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:31.589529 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1111/2200] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:47:31 2022] PUT /identity/v3/projects/c1254f0335304a4fa2a9aca11fe59dc5/users/e8ecdb56dd02499ab0d0cd1e0bc31fea/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:31.604671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c873b2-7696-47bd-a4d0-7ee901d77a8f None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:31.607666 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c873b2-7696-47bd-a4d0-7ee901d77a8f None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:31.608060 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c873b2-7696-47bd-a4d0-7ee901d77a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:31.676734 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-70c873b2-7696-47bd-a4d0-7ee901d77a8f None None] MFA Rules not processed for user `e8ecdb56dd02499ab0d0cd1e0bc31fea`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:31.678068 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-70c873b2-7696-47bd-a4d0-7ee901d77a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:31.853055 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1090/2201] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:31 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:31.870430 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efba2f3-dc24-4ba0-bf72-6dab465e3206 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:31.873270 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-0efba2f3-dc24-4ba0-bf72-6dab465e3206 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:31.971240 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efba2f3-dc24-4ba0-bf72-6dab465e3206 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:31.976185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20550a32-2da6-4901-8353-80a636f9f989 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:31.976854 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20550a32-2da6-4901-8353-80a636f9f989 service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:31.977156 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20550a32-2da6-4901-8353-80a636f9f989 service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:31.977440 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20550a32-2da6-4901-8353-80a636f9f989 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:31.987614 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20550a32-2da6-4901-8353-80a636f9f989 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:31.991273 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20550a32-2da6-4901-8353-80a636f9f989 service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:32.017030 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1112/2202] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:47:31 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:47:32.159074 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c873b2-7696-47bd-a4d0-7ee901d77a8f None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:32.172456 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c873b2-7696-47bd-a4d0-7ee901d77a8f None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:32.182738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f797e11f-14de-44ae-b744-b91959a3854e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:32.183415 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797e11f-14de-44ae-b744-b91959a3854e service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:32.183704 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797e11f-14de-44ae-b744-b91959a3854e service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:32.183988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797e11f-14de-44ae-b744-b91959a3854e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:32.204657 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:32.365932 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f797e11f-14de-44ae-b744-b91959a3854e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:32.367450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f797e11f-14de-44ae-b744-b91959a3854e service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:32.384448 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1091/2203] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:47:32 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:47:32.416657 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20550a32-2da6-4901-8353-80a636f9f989 service cinder] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:32.418466 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-20550a32-2da6-4901-8353-80a636f9f989 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:32.523444 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20550a32-2da6-4901-8353-80a636f9f989 service cinder] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:32.532042 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b5d68-391a-47b1-b244-099f73cfe855 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:32.533563 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b5d68-391a-47b1-b244-099f73cfe855 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:32.534212 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b5d68-391a-47b1-b244-099f73cfe855 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:32.535911 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b5d68-391a-47b1-b244-099f73cfe855 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:32.541435 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-372b5d68-391a-47b1-b244-099f73cfe855 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:32.669763 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372b5d68-391a-47b1-b244-099f73cfe855 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:32.671617 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372b5d68-391a-47b1-b244-099f73cfe855 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:32.704717 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1113/2204] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:47:32 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 13:47:35.959326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d60bb-288b-4b27-b7fb-3ab748cd8111 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:35.959855 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d60bb-288b-4b27-b7fb-3ab748cd8111 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:35.960212 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d60bb-288b-4b27-b7fb-3ab748cd8111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:35.995492 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-ae3d60bb-288b-4b27-b7fb-3ab748cd8111 None None] MFA Rules not processed for user `7dbe70ba377644ba9c146a092655b02e`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:35.997189 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-ae3d60bb-288b-4b27-b7fb-3ab748cd8111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:36.094945 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1092/2205] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:35 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:36.108773 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b5d68-391a-47b1-b244-099f73cfe855 service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:36.111232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-372b5d68-391a-47b1-b244-099f73cfe855 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:36.164976 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:36.233417 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b5d68-391a-47b1-b244-099f73cfe855 service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:36.243607 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717cdb59-a7a6-4cd8-8571-ffab48b0994b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4015c5ca2b8a438288820cd003f9a308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:36.244719 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717cdb59-a7a6-4cd8-8571-ffab48b0994b service glance] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:36.245261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717cdb59-a7a6-4cd8-8571-ffab48b0994b service glance] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:36.245716 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717cdb59-a7a6-4cd8-8571-ffab48b0994b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:36.248036 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-717cdb59-a7a6-4cd8-8571-ffab48b0994b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:36.393813 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717cdb59-a7a6-4cd8-8571-ffab48b0994b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:36.395811 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717cdb59-a7a6-4cd8-8571-ffab48b0994b service glance] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:36.407555 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1114/2206] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:47:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:47:36.647882 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45ea11-17c9-412b-b8eb-203f07d79288 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:36.648560 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45ea11-17c9-412b-b8eb-203f07d79288 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:36.648923 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45ea11-17c9-412b-b8eb-203f07d79288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:36.668738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-4f45ea11-17c9-412b-b8eb-203f07d79288 None None] MFA Rules not processed for user `c4535134181e489fb532f617264d5855`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:36.670114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-4f45ea11-17c9-412b-b8eb-203f07d79288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:36.797533 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1093/2207] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Jul 6 13:47:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:36.860525 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717cdb59-a7a6-4cd8-8571-ffab48b0994b service glance] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:36.862032 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-717cdb59-a7a6-4cd8-8571-ffab48b0994b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:37.018596 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717cdb59-a7a6-4cd8-8571-ffab48b0994b service glance] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:37.024939 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e5db19-4929-43ef-b0df-063e957c57ec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c1bea05c37646b0a05fa8dda0367ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:37.025652 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5db19-4929-43ef-b0df-063e957c57ec service swift] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:37.025940 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5db19-4929-43ef-b0df-063e957c57ec service swift] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:37.026224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5db19-4929-43ef-b0df-063e957c57ec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:37.045890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e5db19-4929-43ef-b0df-063e957c57ec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:37.068912 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e5db19-4929-43ef-b0df-063e957c57ec service swift] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:37.089325 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1115/2208] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jul 6 13:47:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 13:47:46.784894 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=82 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:47.082204 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=69 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:48.736557 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45ea11-17c9-412b-b8eb-203f07d79288 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:48.739160 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-4f45ea11-17c9-412b-b8eb-203f07d79288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:48.929365 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45ea11-17c9-412b-b8eb-203f07d79288 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:48.934178 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6881e-0d79-4a7d-bdca-dff409e19032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:48.935278 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6881e-0d79-4a7d-bdca-dff409e19032 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:48.935631 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6881e-0d79-4a7d-bdca-dff409e19032 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:48.935932 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6881e-0d79-4a7d-bdca-dff409e19032 admin admin] PATH_INFO: `/v3/users/3495c193a38b47789324dd35acde594f` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:48.938074 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c6881e-0d79-4a7d-bdca-dff409e19032 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3495c193a38b47789324dd35acde594f)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:48.941357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c6881e-0d79-4a7d-bdca-dff409e19032 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:49.018148 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-e7c6881e-0d79-4a7d-bdca-dff409e19032 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3495c193a38b47789324dd35acde594f'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:49.030022 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-e7c6881e-0d79-4a7d-bdca-dff409e19032 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3495c193a38b47789324dd35acde594f'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:49.031041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-e7c6881e-0d79-4a7d-bdca-dff409e19032 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3495c193a38b47789324dd35acde594f'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:49.043140 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-e7c6881e-0d79-4a7d-bdca-dff409e19032 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3495c193a38b47789324dd35acde594f'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:49.060132 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-e7c6881e-0d79-4a7d-bdca-dff409e19032 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3495c193a38b47789324dd35acde594f'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:49.067465 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1094/2209] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:47:48 2022] DELETE /identity/v3/users/3495c193a38b47789324dd35acde594f => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:49.116974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e52c0-5127-4869-a53f-fb37fa552554 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:49.119292 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e52c0-5127-4869-a53f-fb37fa552554 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:49.119840 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e52c0-5127-4869-a53f-fb37fa552554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:49.141977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-fa1e52c0-5127-4869-a53f-fb37fa552554 None None] MFA Rules not processed for user `19ccda8f7ce447c490910ffa14165c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:49.144083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-fa1e52c0-5127-4869-a53f-fb37fa552554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:49.219044 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1116/2210] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:49.289698 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6881e-0d79-4a7d-bdca-dff409e19032 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:49.294153 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-e7c6881e-0d79-4a7d-bdca-dff409e19032 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:49.325307 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:49.375656 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6881e-0d79-4a7d-bdca-dff409e19032 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:49.381147 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59ba49-2b4c-46f0-9029-91ffb32ecb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:49.382025 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59ba49-2b4c-46f0-9029-91ffb32ecb6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:49.382317 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59ba49-2b4c-46f0-9029-91ffb32ecb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:49.382599 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59ba49-2b4c-46f0-9029-91ffb32ecb6d admin admin] PATH_INFO: `/v3/projects/a9a20fc57e104d31b0b0c93fce8ccb32` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:49.383827 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59ba49-2b4c-46f0-9029-91ffb32ecb6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9a20fc57e104d31b0b0c93fce8ccb32)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:49.386550 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59ba49-2b4c-46f0-9029-91ffb32ecb6d admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:49.465049 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-6b59ba49-2b4c-46f0-9029-91ffb32ecb6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9a20fc57e104d31b0b0c93fce8ccb32', 'request_id': 'req-6b59ba49-2b4c-46f0-9029-91ffb32ecb6d'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:49.473405 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-6b59ba49-2b4c-46f0-9029-91ffb32ecb6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9a20fc57e104d31b0b0c93fce8ccb32', 'request_id': 'req-6b59ba49-2b4c-46f0-9029-91ffb32ecb6d'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:49.484513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-6b59ba49-2b4c-46f0-9029-91ffb32ecb6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9a20fc57e104d31b0b0c93fce8ccb32', 'request_id': 'req-6b59ba49-2b4c-46f0-9029-91ffb32ecb6d'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:49.501376 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-6b59ba49-2b4c-46f0-9029-91ffb32ecb6d admin admin] The token cache is being invalidate because project a9a20fc57e104d31b0b0c93fce8ccb32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:47:49.501762 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-6b59ba49-2b4c-46f0-9029-91ffb32ecb6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:49.502446 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-6b59ba49-2b4c-46f0-9029-91ffb32ecb6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:49.504295 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1095/2211] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:47:49 2022] DELETE /identity/v3/projects/a9a20fc57e104d31b0b0c93fce8ccb32 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:49.508259 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e52c0-5127-4869-a53f-fb37fa552554 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:49.520632 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e52c0-5127-4869-a53f-fb37fa552554 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:49.612402 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf4527-2619-4ed9-888c-4f126d6e535a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:49.613363 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf4527-2619-4ed9-888c-4f126d6e535a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:49.613799 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf4527-2619-4ed9-888c-4f126d6e535a admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:49.614083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf4527-2619-4ed9-888c-4f126d6e535a admin admin] PATH_INFO: `/v3/users/54065e9b627f4f26b120f1cc7fd169b5` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:49.615435 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bf4527-2619-4ed9-888c-4f126d6e535a admin admin] RBAC: Authorizing `identity:delete_user(user_id=54065e9b627f4f26b120f1cc7fd169b5)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:49.617472 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bf4527-2619-4ed9-888c-4f126d6e535a admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:49.702450 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c8bf4527-2619-4ed9-888c-4f126d6e535a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54065e9b627f4f26b120f1cc7fd169b5'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:49.711335 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59ba49-2b4c-46f0-9029-91ffb32ecb6d admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:49.713266 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-6b59ba49-2b4c-46f0-9029-91ffb32ecb6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:49.716300 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c8bf4527-2619-4ed9-888c-4f126d6e535a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54065e9b627f4f26b120f1cc7fd169b5'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:49.724631 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c8bf4527-2619-4ed9-888c-4f126d6e535a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54065e9b627f4f26b120f1cc7fd169b5'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:49.725602 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c8bf4527-2619-4ed9-888c-4f126d6e535a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54065e9b627f4f26b120f1cc7fd169b5'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:49.739913 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c8bf4527-2619-4ed9-888c-4f126d6e535a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54065e9b627f4f26b120f1cc7fd169b5'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:49.740351 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59ba49-2b4c-46f0-9029-91ffb32ecb6d admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:49.746439 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39f7dca-59db-447b-a2ed-9c7a3bfa9c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:49.747021 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1117/2212] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:47:49 2022] DELETE /identity/v3/users/54065e9b627f4f26b120f1cc7fd169b5 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:49.749518 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f7dca-59db-447b-a2ed-9c7a3bfa9c33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:49.749875 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f7dca-59db-447b-a2ed-9c7a3bfa9c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:49.750159 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f7dca-59db-447b-a2ed-9c7a3bfa9c33 admin admin] PATH_INFO: `/v3/users/f18b2eb526ed4085a6ea694198b56d9b` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:49.751428 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39f7dca-59db-447b-a2ed-9c7a3bfa9c33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f18b2eb526ed4085a6ea694198b56d9b)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:49.754177 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39f7dca-59db-447b-a2ed-9c7a3bfa9c33 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:49.813884 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a39f7dca-59db-447b-a2ed-9c7a3bfa9c33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f18b2eb526ed4085a6ea694198b56d9b'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:49.824262 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a39f7dca-59db-447b-a2ed-9c7a3bfa9c33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f18b2eb526ed4085a6ea694198b56d9b'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:49.824938 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a39f7dca-59db-447b-a2ed-9c7a3bfa9c33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f18b2eb526ed4085a6ea694198b56d9b'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:49.834983 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a39f7dca-59db-447b-a2ed-9c7a3bfa9c33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f18b2eb526ed4085a6ea694198b56d9b'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:49.848842 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a39f7dca-59db-447b-a2ed-9c7a3bfa9c33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f18b2eb526ed4085a6ea694198b56d9b'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:49.855697 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1096/2213] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:47:49 2022] DELETE /identity/v3/users/f18b2eb526ed4085a6ea694198b56d9b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:49.860144 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a819458-e294-429c-a6ad-70157e6fe315 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:49.860583 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a819458-e294-429c-a6ad-70157e6fe315 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:49.860583 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a819458-e294-429c-a6ad-70157e6fe315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:49.890481 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-3a819458-e294-429c-a6ad-70157e6fe315 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:49.891709 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-3a819458-e294-429c-a6ad-70157e6fe315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:49.908648 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:49.996851 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39f7dca-59db-447b-a2ed-9c7a3bfa9c33 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:49.998441 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a39f7dca-59db-447b-a2ed-9c7a3bfa9c33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:50.013064 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1118/2214] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:50.023978 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:50.026788 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a819458-e294-429c-a6ad-70157e6fe315 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:50.030549 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-3a819458-e294-429c-a6ad-70157e6fe315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:50.083188 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39f7dca-59db-447b-a2ed-9c7a3bfa9c33 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:50.088190 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49978acb-8bfc-4ccf-ad78-10c34fc91dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:50.089178 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49978acb-8bfc-4ccf-ad78-10c34fc91dc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:50.089567 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49978acb-8bfc-4ccf-ad78-10c34fc91dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:50.089971 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49978acb-8bfc-4ccf-ad78-10c34fc91dc4 admin admin] PATH_INFO: `/v3/projects/99d3842d090b45ce98a44f21aac8b464` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:50.091413 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49978acb-8bfc-4ccf-ad78-10c34fc91dc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99d3842d090b45ce98a44f21aac8b464)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:50.093511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49978acb-8bfc-4ccf-ad78-10c34fc91dc4 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:50.147148 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-49978acb-8bfc-4ccf-ad78-10c34fc91dc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99d3842d090b45ce98a44f21aac8b464', 'request_id': 'req-49978acb-8bfc-4ccf-ad78-10c34fc91dc4'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:50.149613 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a819458-e294-429c-a6ad-70157e6fe315 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:50.156227 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-49978acb-8bfc-4ccf-ad78-10c34fc91dc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99d3842d090b45ce98a44f21aac8b464', 'request_id': 'req-49978acb-8bfc-4ccf-ad78-10c34fc91dc4'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:50.170068 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-49978acb-8bfc-4ccf-ad78-10c34fc91dc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99d3842d090b45ce98a44f21aac8b464', 'request_id': 'req-49978acb-8bfc-4ccf-ad78-10c34fc91dc4'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:50.182525 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-49978acb-8bfc-4ccf-ad78-10c34fc91dc4 admin admin] The token cache is being invalidate because project 99d3842d090b45ce98a44f21aac8b464 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:47:50.182816 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-49978acb-8bfc-4ccf-ad78-10c34fc91dc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:50.183479 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-49978acb-8bfc-4ccf-ad78-10c34fc91dc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:50.185061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1097/2215] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:47:49 2022] DELETE /identity/v3/projects/99d3842d090b45ce98a44f21aac8b464 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:50.190241 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe50ae5-7f28-412e-8c28-e556af94ef3b None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:50.190780 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe50ae5-7f28-412e-8c28-e556af94ef3b None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:50.191193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe50ae5-7f28-412e-8c28-e556af94ef3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:50.208929 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd2a85-93f6-4d18-ab93-447be9cc3367 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:50.209627 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd2a85-93f6-4d18-ab93-447be9cc3367 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:50.210706 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd2a85-93f6-4d18-ab93-447be9cc3367 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:50.210706 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd2a85-93f6-4d18-ab93-447be9cc3367 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:50.212249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-a9fd2a85-93f6-4d18-ab93-447be9cc3367 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:50.234841 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-ebe50ae5-7f28-412e-8c28-e556af94ef3b None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:50.236181 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-ebe50ae5-7f28-412e-8c28-e556af94ef3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:50.279337 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:50.307215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fd2a85-93f6-4d18-ab93-447be9cc3367 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:50.308959 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fd2a85-93f6-4d18-ab93-447be9cc3367 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:50.328342 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1119/2216] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:47:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:47:50.329769 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1098/2217] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:50.397319 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf4171-d9a6-46c9-9816-f141c7d567e1 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:50.398834 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf4171-d9a6-46c9-9816-f141c7d567e1 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:50.398834 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf4171-d9a6-46c9-9816-f141c7d567e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:50.426394 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-1ecf4171-d9a6-46c9-9816-f141c7d567e1 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:50.428189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-1ecf4171-d9a6-46c9-9816-f141c7d567e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:50.437458 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1120/2218] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:50.444433 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe50ae5-7f28-412e-8c28-e556af94ef3b None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:50.458548 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe50ae5-7f28-412e-8c28-e556af94ef3b None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:50.464276 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8a7de-ee7e-461e-99fa-403efcd8b24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:50.465039 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8a7de-ee7e-461e-99fa-403efcd8b24e admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:50.465515 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8a7de-ee7e-461e-99fa-403efcd8b24e admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:50.465677 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8a7de-ee7e-461e-99fa-403efcd8b24e admin admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:50.466791 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8a7de-ee7e-461e-99fa-403efcd8b24e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:47:50.467286 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8a7de-ee7e-461e-99fa-403efcd8b24e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:50.472723 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8a7de-ee7e-461e-99fa-403efcd8b24e admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:50.488929 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1099/2219] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:47:50.493364 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf4171-d9a6-46c9-9816-f141c7d567e1 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:50.503940 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf4171-d9a6-46c9-9816-f141c7d567e1 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:50.509103 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa511427-f36b-4a9c-bf2c-47ee9fe07f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:50.510436 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa511427-f36b-4a9c-bf2c-47ee9fe07f74 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:50.510878 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa511427-f36b-4a9c-bf2c-47ee9fe07f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:50.511307 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa511427-f36b-4a9c-bf2c-47ee9fe07f74 admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:50.512524 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa511427-f36b-4a9c-bf2c-47ee9fe07f74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:50.514702 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa511427-f36b-4a9c-bf2c-47ee9fe07f74 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:50.556688 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1121/2220] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:47:50 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:50.557945 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8a7de-ee7e-461e-99fa-403efcd8b24e admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:50.572372 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8a7de-ee7e-461e-99fa-403efcd8b24e admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:50.600299 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:50.655110 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b8d99-f6de-4dca-aece-5ca78dfc626c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:50.655903 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b8d99-f6de-4dca-aece-5ca78dfc626c admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:50.656306 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b8d99-f6de-4dca-aece-5ca78dfc626c admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:50.656649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b8d99-f6de-4dca-aece-5ca78dfc626c admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:50.657377 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b8d99-f6de-4dca-aece-5ca78dfc626c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:50.659969 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b8d99-f6de-4dca-aece-5ca78dfc626c admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:50.685034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1100/2221] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:47:50 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:50.685195 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa511427-f36b-4a9c-bf2c-47ee9fe07f74 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:50.686466 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-aa511427-f36b-4a9c-bf2c-47ee9fe07f74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:50.686604 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b8d99-f6de-4dca-aece-5ca78dfc626c admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:50.710481 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b8d99-f6de-4dca-aece-5ca78dfc626c admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:50.716442 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa511427-f36b-4a9c-bf2c-47ee9fe07f74 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:50.716875 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b2ca7-ec60-4c50-b69b-945f4c3c6b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:50.717661 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b2ca7-ec60-4c50-b69b-945f4c3c6b49 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:50.717945 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b2ca7-ec60-4c50-b69b-945f4c3c6b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:50.718223 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b2ca7-ec60-4c50-b69b-945f4c3c6b49 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:50.719279 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b2ca7-ec60-4c50-b69b-945f4c3c6b49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:50.720865 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b2ca7-ec60-4c50-b69b-945f4c3c6b49 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:50.726655 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19fefa8-74a7-413a-b6fc-295bd7dac0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:50.729632 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fefa8-74a7-413a-b6fc-295bd7dac0d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:50.729929 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fefa8-74a7-413a-b6fc-295bd7dac0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:50.730215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fefa8-74a7-413a-b6fc-295bd7dac0d5 admin admin] PATH_INFO: `/v3/projects/93feebcc3429463aaa122941fa66f29c` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:50.732082 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19fefa8-74a7-413a-b6fc-295bd7dac0d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93feebcc3429463aaa122941fa66f29c)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:50.734045 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19fefa8-74a7-413a-b6fc-295bd7dac0d5 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:50.739116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1101/2222] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:47:50.744377 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b2ca7-ec60-4c50-b69b-945f4c3c6b49 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:50.758019 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b2ca7-ec60-4c50-b69b-945f4c3c6b49 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:50.774580 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c2df7-1b0f-4ac8-934f-1494d008e213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:50.775580 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c2df7-1b0f-4ac8-934f-1494d008e213 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:50.775873 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c2df7-1b0f-4ac8-934f-1494d008e213 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:50.776152 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c2df7-1b0f-4ac8-934f-1494d008e213 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:50.777344 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c2df7-1b0f-4ac8-934f-1494d008e213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:50.780147 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c2df7-1b0f-4ac8-934f-1494d008e213 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:50.816459 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c19fefa8-74a7-413a-b6fc-295bd7dac0d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93feebcc3429463aaa122941fa66f29c', 'request_id': 'req-c19fefa8-74a7-413a-b6fc-295bd7dac0d5'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:50.819632 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1102/2223] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:47:50.826321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c19fefa8-74a7-413a-b6fc-295bd7dac0d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93feebcc3429463aaa122941fa66f29c', 'request_id': 'req-c19fefa8-74a7-413a-b6fc-295bd7dac0d5'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:50.834634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c19fefa8-74a7-413a-b6fc-295bd7dac0d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93feebcc3429463aaa122941fa66f29c', 'request_id': 'req-c19fefa8-74a7-413a-b6fc-295bd7dac0d5'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:50.835953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c2df7-1b0f-4ac8-934f-1494d008e213 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:50.851466 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c19fefa8-74a7-413a-b6fc-295bd7dac0d5 admin admin] The token cache is being invalidate because project 93feebcc3429463aaa122941fa66f29c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:47:50.851760 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c19fefa8-74a7-413a-b6fc-295bd7dac0d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:50.852453 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c19fefa8-74a7-413a-b6fc-295bd7dac0d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:50.856877 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1122/2224] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:47:50 2022] DELETE /identity/v3/projects/93feebcc3429463aaa122941fa66f29c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:50.857299 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c2df7-1b0f-4ac8-934f-1494d008e213 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:50.857981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19fefa8-74a7-413a-b6fc-295bd7dac0d5 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:50.859379 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-c19fefa8-74a7-413a-b6fc-295bd7dac0d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:50.931733 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19fefa8-74a7-413a-b6fc-295bd7dac0d5 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:50.932293 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a1b03-47a2-4332-abb9-a46a5a839ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:50.933218 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a1b03-47a2-4332-abb9-a46a5a839ca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:50.933503 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a1b03-47a2-4332-abb9-a46a5a839ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:50.933778 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a1b03-47a2-4332-abb9-a46a5a839ca2 admin admin] PATH_INFO: `/v3/projects/a004d82bf845459c8839f5590efc16ba/users/42064b0cc72e4112912005c4335b8146/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:50.937288 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa14253c-1393-4b9e-96f6-1f51d7c19aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:50.938246 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14253c-1393-4b9e-96f6-1f51d7c19aaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:50.938534 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14253c-1393-4b9e-96f6-1f51d7c19aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:50.938836 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14253c-1393-4b9e-96f6-1f51d7c19aaa admin admin] PATH_INFO: `/v3/users/e8ecdb56dd02499ab0d0cd1e0bc31fea` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:50.940058 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa14253c-1393-4b9e-96f6-1f51d7c19aaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8ecdb56dd02499ab0d0cd1e0bc31fea)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:50.942094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa14253c-1393-4b9e-96f6-1f51d7c19aaa admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:50.951672 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a1b03-47a2-4332-abb9-a46a5a839ca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a004d82bf845459c8839f5590efc16ba, user_id=42064b0cc72e4112912005c4335b8146, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:50.954639 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a1b03-47a2-4332-abb9-a46a5a839ca2 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:50.969054 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1103/2225] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:47:50 2022] PUT /identity/v3/projects/a004d82bf845459c8839f5590efc16ba/users/42064b0cc72e4112912005c4335b8146/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:50.980780 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a94fe29-d625-47f7-a5f1-72ac11e632c5 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:50.981133 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a94fe29-d625-47f7-a5f1-72ac11e632c5 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:50.981418 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a94fe29-d625-47f7-a5f1-72ac11e632c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:50.997790 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=84,keystone:INSERT=2,keystone:DELETE=14 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:51.002243 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-fa14253c-1393-4b9e-96f6-1f51d7c19aaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8ecdb56dd02499ab0d0cd1e0bc31fea'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:51.013399 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-2a94fe29-d625-47f7-a5f1-72ac11e632c5 None None] MFA Rules not processed for user `42064b0cc72e4112912005c4335b8146`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:51.014814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-2a94fe29-d625-47f7-a5f1-72ac11e632c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:51.015375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-fa14253c-1393-4b9e-96f6-1f51d7c19aaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8ecdb56dd02499ab0d0cd1e0bc31fea'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:51.025529 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-fa14253c-1393-4b9e-96f6-1f51d7c19aaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8ecdb56dd02499ab0d0cd1e0bc31fea'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:51.026366 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-fa14253c-1393-4b9e-96f6-1f51d7c19aaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8ecdb56dd02499ab0d0cd1e0bc31fea'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:51.042823 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-fa14253c-1393-4b9e-96f6-1f51d7c19aaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8ecdb56dd02499ab0d0cd1e0bc31fea'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:51.050605 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1123/2226] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:47:50 2022] DELETE /identity/v3/users/e8ecdb56dd02499ab0d0cd1e0bc31fea => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:51.106229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1104/2227] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:51.118107 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa14253c-1393-4b9e-96f6-1f51d7c19aaa admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:51.119640 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-fa14253c-1393-4b9e-96f6-1f51d7c19aaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:51.181884 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa14253c-1393-4b9e-96f6-1f51d7c19aaa admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:51.186041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a96e9d-cd33-4b14-ac0e-c68c9af7eb16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:51.186688 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a96e9d-cd33-4b14-ac0e-c68c9af7eb16 service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:51.186967 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a96e9d-cd33-4b14-ac0e-c68c9af7eb16 service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:51.187242 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a96e9d-cd33-4b14-ac0e-c68c9af7eb16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:51.250973 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a96e9d-cd33-4b14-ac0e-c68c9af7eb16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:51.252440 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a96e9d-cd33-4b14-ac0e-c68c9af7eb16 service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:51.262953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1124/2228] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:47:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:47:51.376248 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a94fe29-d625-47f7-a5f1-72ac11e632c5 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:51.379009 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-2a94fe29-d625-47f7-a5f1-72ac11e632c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:51.385802 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:51.483139 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a94fe29-d625-47f7-a5f1-72ac11e632c5 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:51.569491 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e16f2db-3a21-4e4a-a083-50c9c23c0c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:51.570556 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16f2db-3a21-4e4a-a083-50c9c23c0c7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:51.570844 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16f2db-3a21-4e4a-a083-50c9c23c0c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:51.571130 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16f2db-3a21-4e4a-a083-50c9c23c0c7c admin admin] PATH_INFO: `/v3/projects/c1254f0335304a4fa2a9aca11fe59dc5` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:51.572488 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e16f2db-3a21-4e4a-a083-50c9c23c0c7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1254f0335304a4fa2a9aca11fe59dc5)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:51.575114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e16f2db-3a21-4e4a-a083-50c9c23c0c7c admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:51.628688 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-1e16f2db-3a21-4e4a-a083-50c9c23c0c7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1254f0335304a4fa2a9aca11fe59dc5', 'request_id': 'req-1e16f2db-3a21-4e4a-a083-50c9c23c0c7c'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:51.635075 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-1e16f2db-3a21-4e4a-a083-50c9c23c0c7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1254f0335304a4fa2a9aca11fe59dc5', 'request_id': 'req-1e16f2db-3a21-4e4a-a083-50c9c23c0c7c'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:51.643188 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-1e16f2db-3a21-4e4a-a083-50c9c23c0c7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1254f0335304a4fa2a9aca11fe59dc5', 'request_id': 'req-1e16f2db-3a21-4e4a-a083-50c9c23c0c7c'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:51.655077 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-1e16f2db-3a21-4e4a-a083-50c9c23c0c7c admin admin] The token cache is being invalidate because project c1254f0335304a4fa2a9aca11fe59dc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:47:51.655392 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-1e16f2db-3a21-4e4a-a083-50c9c23c0c7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:51.656052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-1e16f2db-3a21-4e4a-a083-50c9c23c0c7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:47:51.657637 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1105/2229] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:47:51 2022] DELETE /identity/v3/projects/c1254f0335304a4fa2a9aca11fe59dc5 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:51.664348 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e016d3-17b7-4383-9f73-24fa9df78643 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:51.664674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e016d3-17b7-4383-9f73-24fa9df78643 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:51.664974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e016d3-17b7-4383-9f73-24fa9df78643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:51.683173 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-a5e016d3-17b7-4383-9f73-24fa9df78643 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:51.684351 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-a5e016d3-17b7-4383-9f73-24fa9df78643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:51.741989 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1125/2230] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:51.749786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931acf2d-0837-4f29-97eb-858fc6eebc37 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:51.750251 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931acf2d-0837-4f29-97eb-858fc6eebc37 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:51.750349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931acf2d-0837-4f29-97eb-858fc6eebc37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:51.769363 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-931acf2d-0837-4f29-97eb-858fc6eebc37 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:51.770589 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-931acf2d-0837-4f29-97eb-858fc6eebc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:51.780015 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1106/2231] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:51.785375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e016d3-17b7-4383-9f73-24fa9df78643 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:51.796284 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e016d3-17b7-4383-9f73-24fa9df78643 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:51.803308 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c00375-4e25-4004-ba06-fe103ec62bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:51.803994 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c00375-4e25-4004-ba06-fe103ec62bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:51.804281 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c00375-4e25-4004-ba06-fe103ec62bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:51.804561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c00375-4e25-4004-ba06-fe103ec62bc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:51.805478 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c00375-4e25-4004-ba06-fe103ec62bc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:47:51.805661 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c00375-4e25-4004-ba06-fe103ec62bc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:51.807669 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c00375-4e25-4004-ba06-fe103ec62bc2 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:51.823514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1126/2232] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:47:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:47:51.828088 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931acf2d-0837-4f29-97eb-858fc6eebc37 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:51.838310 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931acf2d-0837-4f29-97eb-858fc6eebc37 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:51.843136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df596a-a440-4a45-a173-8f3c88dcc3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:51.843793 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df596a-a440-4a45-a173-8f3c88dcc3a5 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:51.844080 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df596a-a440-4a45-a173-8f3c88dcc3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:51.844685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df596a-a440-4a45-a173-8f3c88dcc3a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:51.845483 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df596a-a440-4a45-a173-8f3c88dcc3a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:51.848452 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df596a-a440-4a45-a173-8f3c88dcc3a5 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:51.878325 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1107/2233] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:47:51 2022] POST /identity/v3/projects => generated 399 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:51.882702 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c00375-4e25-4004-ba06-fe103ec62bc2 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:51.889487 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:51.893700 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c00375-4e25-4004-ba06-fe103ec62bc2 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:51.955682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ec039-b277-419e-a3ca-10ddd9cf2222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:51.956822 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ec039-b277-419e-a3ca-10ddd9cf2222 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:51.957145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ec039-b277-419e-a3ca-10ddd9cf2222 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:51.957428 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ec039-b277-419e-a3ca-10ddd9cf2222 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:51.958135 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ec039-b277-419e-a3ca-10ddd9cf2222 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:51.960045 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ec039-b277-419e-a3ca-10ddd9cf2222 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:51.981954 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1127/2234] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:47:51 2022] POST /identity/v3/users => generated 300 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:51.986288 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df596a-a440-4a45-a173-8f3c88dcc3a5 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:51.997311 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df596a-a440-4a45-a173-8f3c88dcc3a5 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:52.002316 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b6b19-b2d4-4fca-9534-ed4a4e96f90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:52.003619 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b6b19-b2d4-4fca-9534-ed4a4e96f90b admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:52.003967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b6b19-b2d4-4fca-9534-ed4a4e96f90b admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:52.004303 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b6b19-b2d4-4fca-9534-ed4a4e96f90b admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:52.005120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3b6b19-b2d4-4fca-9534-ed4a4e96f90b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:52.006705 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3b6b19-b2d4-4fca-9534-ed4a4e96f90b admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:52.012457 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:52.021773 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1108/2235] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:47:52.024451 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ec039-b277-419e-a3ca-10ddd9cf2222 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:52.035227 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ec039-b277-419e-a3ca-10ddd9cf2222 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:52.041395 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02552e31-4d61-46a7-be69-4ca8ddad2949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:52.042278 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02552e31-4d61-46a7-be69-4ca8ddad2949 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:52.042564 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02552e31-4d61-46a7-be69-4ca8ddad2949 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:52.042846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02552e31-4d61-46a7-be69-4ca8ddad2949 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:52.043589 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02552e31-4d61-46a7-be69-4ca8ddad2949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:52.045257 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02552e31-4d61-46a7-be69-4ca8ddad2949 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:52.056311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1128/2236] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:47:52.060221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b6b19-b2d4-4fca-9534-ed4a4e96f90b admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:52.079846 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b6b19-b2d4-4fca-9534-ed4a4e96f90b admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:52.085810 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e873d-28b8-4358-9324-8f2d945986e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:52.086711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e873d-28b8-4358-9324-8f2d945986e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:52.086988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e873d-28b8-4358-9324-8f2d945986e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:52.087298 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e873d-28b8-4358-9324-8f2d945986e3 admin admin] PATH_INFO: `/v3/projects/9eb424a779ca421c95184a3a189b558f/users/fd3dd4d7c753449eaad1d76e69fffe5b/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:52.103998 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e873d-28b8-4358-9324-8f2d945986e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eb424a779ca421c95184a3a189b558f, user_id=fd3dd4d7c753449eaad1d76e69fffe5b, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:52.107134 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e873d-28b8-4358-9324-8f2d945986e3 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:52.123407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1109/2237] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:47:52 2022] PUT /identity/v3/projects/9eb424a779ca421c95184a3a189b558f/users/fd3dd4d7c753449eaad1d76e69fffe5b/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:52.137215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49621dc-5d0d-4403-b80a-21c8be32b524 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:52.137215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49621dc-5d0d-4403-b80a-21c8be32b524 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:52.137215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49621dc-5d0d-4403-b80a-21c8be32b524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:52.162033 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-e49621dc-5d0d-4403-b80a-21c8be32b524 None None] MFA Rules not processed for user `fd3dd4d7c753449eaad1d76e69fffe5b`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:52.163228 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-e49621dc-5d0d-4403-b80a-21c8be32b524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:52.227934 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1129/2238] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:52 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:47:52.232343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e873d-28b8-4358-9324-8f2d945986e3 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:52.294208 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e873d-28b8-4358-9324-8f2d945986e3 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:52.298847 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a28d80-8df8-49d7-938a-33dc5c83ba1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:52.300088 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a28d80-8df8-49d7-938a-33dc5c83ba1f admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:52.300433 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a28d80-8df8-49d7-938a-33dc5c83ba1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:52.301154 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a28d80-8df8-49d7-938a-33dc5c83ba1f admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:52.301888 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a28d80-8df8-49d7-938a-33dc5c83ba1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:52.304299 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a28d80-8df8-49d7-938a-33dc5c83ba1f admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:52.334434 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1110/2239] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:47:52 2022] POST /identity/v3/projects => generated 399 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:52.336370 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49621dc-5d0d-4403-b80a-21c8be32b524 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:52.348484 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49621dc-5d0d-4403-b80a-21c8be32b524 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:52.425942 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e17146-32a8-4f4c-b02c-df0e733fc3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:52.426636 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e17146-32a8-4f4c-b02c-df0e733fc3e6 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:52.426927 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e17146-32a8-4f4c-b02c-df0e733fc3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:52.427210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e17146-32a8-4f4c-b02c-df0e733fc3e6 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:52.427915 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e17146-32a8-4f4c-b02c-df0e733fc3e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:52.429903 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e17146-32a8-4f4c-b02c-df0e733fc3e6 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:52.456054 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1130/2240] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:47:52 2022] POST /identity/v3/users => generated 306 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:47:52.460107 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a28d80-8df8-49d7-938a-33dc5c83ba1f admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:52.471792 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a28d80-8df8-49d7-938a-33dc5c83ba1f admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:52.476936 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6880e084-4a03-49b8-913a-c665ea72071c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:52.477638 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880e084-4a03-49b8-913a-c665ea72071c admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:52.477923 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880e084-4a03-49b8-913a-c665ea72071c admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:52.478200 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880e084-4a03-49b8-913a-c665ea72071c admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:52.478927 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6880e084-4a03-49b8-913a-c665ea72071c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:52.480604 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6880e084-4a03-49b8-913a-c665ea72071c admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:52.492658 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1111/2241] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:47:52.496760 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e17146-32a8-4f4c-b02c-df0e733fc3e6 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:52.501891 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:52.508859 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e17146-32a8-4f4c-b02c-df0e733fc3e6 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:52.514029 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e18325-7833-4c57-bbd3-a914ed061469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:52.515061 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e18325-7833-4c57-bbd3-a914ed061469 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:52.515312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e18325-7833-4c57-bbd3-a914ed061469 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:52.515590 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e18325-7833-4c57-bbd3-a914ed061469 admin admin] PATH_INFO: `/v3/projects/c32dd3295f4445669e4ec1cccbea13e2/users/5cc2da1f46d0407fbda259a694f5d81f/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:52.533973 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e18325-7833-4c57-bbd3-a914ed061469 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c32dd3295f4445669e4ec1cccbea13e2, user_id=5cc2da1f46d0407fbda259a694f5d81f, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:52.537547 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e18325-7833-4c57-bbd3-a914ed061469 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:52.564228 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1131/2242] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:47:52 2022] PUT /identity/v3/projects/c32dd3295f4445669e4ec1cccbea13e2/users/5cc2da1f46d0407fbda259a694f5d81f/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:52.574530 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6880e084-4a03-49b8-913a-c665ea72071c admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:52.585295 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6880e084-4a03-49b8-913a-c665ea72071c admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:52.678649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1e002-df5c-4a9f-9752-7a7be702fc98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:52.679603 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1e002-df5c-4a9f-9752-7a7be702fc98 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:52.680291 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1e002-df5c-4a9f-9752-7a7be702fc98 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:52.680854 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1e002-df5c-4a9f-9752-7a7be702fc98 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:52.685565 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1e002-df5c-4a9f-9752-7a7be702fc98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:52.688227 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1e002-df5c-4a9f-9752-7a7be702fc98 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:52.706541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1112/2243] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:47:52.707262 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e18325-7833-4c57-bbd3-a914ed061469 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:52.721275 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e18325-7833-4c57-bbd3-a914ed061469 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:52.727317 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc81d84-c7f0-4f14-8f7b-8dbcb0a68e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:52.728223 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc81d84-c7f0-4f14-8f7b-8dbcb0a68e1f admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:52.728505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc81d84-c7f0-4f14-8f7b-8dbcb0a68e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:52.728781 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc81d84-c7f0-4f14-8f7b-8dbcb0a68e1f admin admin] PATH_INFO: `/v3/projects/c32dd3295f4445669e4ec1cccbea13e2/users/5cc2da1f46d0407fbda259a694f5d81f/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:52.731733 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc81d84-c7f0-4f14-8f7b-8dbcb0a68e1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c32dd3295f4445669e4ec1cccbea13e2, user_id=5cc2da1f46d0407fbda259a694f5d81f, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:52.737213 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc81d84-c7f0-4f14-8f7b-8dbcb0a68e1f admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:52.753158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1132/2244] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:47:52 2022] PUT /identity/v3/projects/c32dd3295f4445669e4ec1cccbea13e2/users/5cc2da1f46d0407fbda259a694f5d81f/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:47:52.772024 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cb7be-2a8d-45aa-9b01-5e9edcb25ba3 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:52.772539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cb7be-2a8d-45aa-9b01-5e9edcb25ba3 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:52.772657 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cb7be-2a8d-45aa-9b01-5e9edcb25ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:52.807741 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-fc3cb7be-2a8d-45aa-9b01-5e9edcb25ba3 None None] MFA Rules not processed for user `5cc2da1f46d0407fbda259a694f5d81f`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:52.810796 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-fc3cb7be-2a8d-45aa-9b01-5e9edcb25ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:52.859112 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:47:52.921916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1113/2245] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:47:52 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:47:52.941905 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc81d84-c7f0-4f14-8f7b-8dbcb0a68e1f admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:52.943641 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-afc81d84-c7f0-4f14-8f7b-8dbcb0a68e1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:53.026570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc81d84-c7f0-4f14-8f7b-8dbcb0a68e1f admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:53.030416 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3bcc9-ad16-4c52-96c1-0c717c3919de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:53.033804 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3bcc9-ad16-4c52-96c1-0c717c3919de service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:53.036413 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3bcc9-ad16-4c52-96c1-0c717c3919de service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:53.036873 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3bcc9-ad16-4c52-96c1-0c717c3919de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:53.056567 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f3bcc9-ad16-4c52-96c1-0c717c3919de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:53.058820 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f3bcc9-ad16-4c52-96c1-0c717c3919de service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:53.071873 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1133/2246] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:47:52 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:47:53.159736 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3cb7be-2a8d-45aa-9b01-5e9edcb25ba3 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:53.175315 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3cb7be-2a8d-45aa-9b01-5e9edcb25ba3 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:53.181078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3871f34-4dde-4b69-979d-b03fe3d4e195 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:53.182617 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3871f34-4dde-4b69-979d-b03fe3d4e195 service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:53.182995 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3871f34-4dde-4b69-979d-b03fe3d4e195 service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:53.183389 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3871f34-4dde-4b69-979d-b03fe3d4e195 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:53.324492 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3871f34-4dde-4b69-979d-b03fe3d4e195 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:53.326560 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3871f34-4dde-4b69-979d-b03fe3d4e195 service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:53.352332 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1114/2247] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:47:53 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:47:53.738730 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab264ada-61ef-442f-85e9-b4341525fcaa None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:53.739176 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab264ada-61ef-442f-85e9-b4341525fcaa None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:53.739476 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab264ada-61ef-442f-85e9-b4341525fcaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:53.760400 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-ab264ada-61ef-442f-85e9-b4341525fcaa None None] MFA Rules not processed for user `c4535134181e489fb532f617264d5855`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:47:53.761642 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-ab264ada-61ef-442f-85e9-b4341525fcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:53.836883 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1134/2248] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Jul 6 13:47:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:47:53.850528 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3871f34-4dde-4b69-979d-b03fe3d4e195 service cinder] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:47:53.852594 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a3871f34-4dde-4b69-979d-b03fe3d4e195 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:47:53.925373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3871f34-4dde-4b69-979d-b03fe3d4e195 service cinder] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:47:53.930568 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee3ad9-3e2d-47d4-ad05-64f2a488a269 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c1bea05c37646b0a05fa8dda0367ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:47:53.931211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee3ad9-3e2d-47d4-ad05-64f2a488a269 service swift] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:47:53.931490 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee3ad9-3e2d-47d4-ad05-64f2a488a269 service swift] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:47:53.931763 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee3ad9-3e2d-47d4-ad05-64f2a488a269 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:47:53.940022 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ee3ad9-3e2d-47d4-ad05-64f2a488a269 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:47:53.942943 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ee3ad9-3e2d-47d4-ad05-64f2a488a269 service swift] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:47:53.951516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1115/2249] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jul 6 13:47:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 13:48:03.828136 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=65,keystone:INSERT=2 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:48:03.948122 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=73 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:48:07.976491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f90e3e-66bb-4fb6-8297-c3686f391573 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:07.977124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f90e3e-66bb-4fb6-8297-c3686f391573 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:07.977124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f90e3e-66bb-4fb6-8297-c3686f391573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:08.011401 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-c5f90e3e-66bb-4fb6-8297-c3686f391573 None None] MFA Rules not processed for user `c4535134181e489fb532f617264d5855`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:48:08.012682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-c5f90e3e-66bb-4fb6-8297-c3686f391573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:08.032839 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1135/2250] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Jul 6 13:48:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:48:08.057164 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee3ad9-3e2d-47d4-ad05-64f2a488a269 service swift] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:08.067862 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee3ad9-3e2d-47d4-ad05-64f2a488a269 service swift] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:08.072904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e57b95-43d4-4814-9355-f312316b28c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c1bea05c37646b0a05fa8dda0367ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:08.073571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e57b95-43d4-4814-9355-f312316b28c5 service swift] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:08.073855 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e57b95-43d4-4814-9355-f312316b28c5 service swift] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:08.074241 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e57b95-43d4-4814-9355-f312316b28c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:08.082696 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e57b95-43d4-4814-9355-f312316b28c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:08.084627 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e57b95-43d4-4814-9355-f312316b28c5 service swift] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:08.100182 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1116/2251] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jul 6 13:48:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 13:48:08.215231 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f90e3e-66bb-4fb6-8297-c3686f391573 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:08.217202 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-c5f90e3e-66bb-4fb6-8297-c3686f391573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:08.304616 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f90e3e-66bb-4fb6-8297-c3686f391573 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:08.311656 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e011ec-d589-4bea-b619-4ed73725eaf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:08.312986 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e011ec-d589-4bea-b619-4ed73725eaf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:08.313884 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e011ec-d589-4bea-b619-4ed73725eaf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:08.314378 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e011ec-d589-4bea-b619-4ed73725eaf0 admin admin] PATH_INFO: `/v3/users/42064b0cc72e4112912005c4335b8146` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:08.316502 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e011ec-d589-4bea-b619-4ed73725eaf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42064b0cc72e4112912005c4335b8146)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:08.319001 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e011ec-d589-4bea-b619-4ed73725eaf0 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:08.391954 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-11e011ec-d589-4bea-b619-4ed73725eaf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42064b0cc72e4112912005c4335b8146'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:08.406885 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-11e011ec-d589-4bea-b619-4ed73725eaf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42064b0cc72e4112912005c4335b8146'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:08.419153 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-11e011ec-d589-4bea-b619-4ed73725eaf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42064b0cc72e4112912005c4335b8146'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:08.420021 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-11e011ec-d589-4bea-b619-4ed73725eaf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42064b0cc72e4112912005c4335b8146'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:08.440021 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-11e011ec-d589-4bea-b619-4ed73725eaf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42064b0cc72e4112912005c4335b8146'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:08.446508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1136/2252] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:48:08 2022] DELETE /identity/v3/users/42064b0cc72e4112912005c4335b8146 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:48:08.451039 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6be53-211b-4ea4-8259-1c48ecc742da None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:08.451326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6be53-211b-4ea4-8259-1c48ecc742da None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:08.451604 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6be53-211b-4ea4-8259-1c48ecc742da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:08.471547 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-b5a6be53-211b-4ea4-8259-1c48ecc742da None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:48:08.472763 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-b5a6be53-211b-4ea4-8259-1c48ecc742da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:08.577366 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1117/2253] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:48:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:48:08.585063 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e011ec-d589-4bea-b619-4ed73725eaf0 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:08.586162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-11e011ec-d589-4bea-b619-4ed73725eaf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:08.749854 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e011ec-d589-4bea-b619-4ed73725eaf0 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:08.763647 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19b572-03e8-4286-874e-73fb30b8e434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:08.764330 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19b572-03e8-4286-874e-73fb30b8e434 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:08.764618 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19b572-03e8-4286-874e-73fb30b8e434 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:08.764898 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19b572-03e8-4286-874e-73fb30b8e434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:08.784068 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19b572-03e8-4286-874e-73fb30b8e434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:08.785390 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19b572-03e8-4286-874e-73fb30b8e434 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:08.795872 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:48:08.817647 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1137/2254] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:48:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:48:09.408888 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6be53-211b-4ea4-8259-1c48ecc742da None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:09.411813 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-b5a6be53-211b-4ea4-8259-1c48ecc742da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:09.441337 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6be53-211b-4ea4-8259-1c48ecc742da None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:09.477822 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb034f-d11b-4ecc-92ed-68f8e4703fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:09.479522 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb034f-d11b-4ecc-92ed-68f8e4703fd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:09.492195 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb034f-d11b-4ecc-92ed-68f8e4703fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:09.492195 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb034f-d11b-4ecc-92ed-68f8e4703fd5 admin admin] PATH_INFO: `/v3/projects/a004d82bf845459c8839f5590efc16ba` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:09.501875 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb034f-d11b-4ecc-92ed-68f8e4703fd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a004d82bf845459c8839f5590efc16ba)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:09.508937 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb034f-d11b-4ecc-92ed-68f8e4703fd5 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:09.598987 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19b572-03e8-4286-874e-73fb30b8e434 service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:09.600491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-bb19b572-03e8-4286-874e-73fb30b8e434 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:09.623366 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19b572-03e8-4286-874e-73fb30b8e434 service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:09.631241 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-fbfb034f-d11b-4ecc-92ed-68f8e4703fd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a004d82bf845459c8839f5590efc16ba', 'request_id': 'req-fbfb034f-d11b-4ecc-92ed-68f8e4703fd5'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:09.649405 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-fbfb034f-d11b-4ecc-92ed-68f8e4703fd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a004d82bf845459c8839f5590efc16ba', 'request_id': 'req-fbfb034f-d11b-4ecc-92ed-68f8e4703fd5'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:09.667484 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-fbfb034f-d11b-4ecc-92ed-68f8e4703fd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a004d82bf845459c8839f5590efc16ba', 'request_id': 'req-fbfb034f-d11b-4ecc-92ed-68f8e4703fd5'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:09.693134 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-fbfb034f-d11b-4ecc-92ed-68f8e4703fd5 admin admin] The token cache is being invalidate because project a004d82bf845459c8839f5590efc16ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:48:09.693712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-fbfb034f-d11b-4ecc-92ed-68f8e4703fd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:09.700726 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-fbfb034f-d11b-4ecc-92ed-68f8e4703fd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:09.706246 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1118/2255] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:48:09 2022] DELETE /identity/v3/projects/a004d82bf845459c8839f5590efc16ba => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:48:09.707747 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dab82a-652a-4ac8-af2f-3af778506344 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:09.708051 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dab82a-652a-4ac8-af2f-3af778506344 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:09.708340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dab82a-652a-4ac8-af2f-3af778506344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:09.710448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f86665-bf02-4515-b2a5-672aacd85761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:09.711378 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f86665-bf02-4515-b2a5-672aacd85761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:09.711670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f86665-bf02-4515-b2a5-672aacd85761 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:09.711950 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f86665-bf02-4515-b2a5-672aacd85761 admin admin] PATH_INFO: `/v3/users/fd3dd4d7c753449eaad1d76e69fffe5b` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:09.713310 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f86665-bf02-4515-b2a5-672aacd85761 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd3dd4d7c753449eaad1d76e69fffe5b)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:09.715278 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f86665-bf02-4515-b2a5-672aacd85761 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:09.785100 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-f5dab82a-652a-4ac8-af2f-3af778506344 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:48:09.787230 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-f5dab82a-652a-4ac8-af2f-3af778506344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:09.792907 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-56f86665-bf02-4515-b2a5-672aacd85761 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd3dd4d7c753449eaad1d76e69fffe5b'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:09.816713 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-56f86665-bf02-4515-b2a5-672aacd85761 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd3dd4d7c753449eaad1d76e69fffe5b'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:09.829620 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-56f86665-bf02-4515-b2a5-672aacd85761 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd3dd4d7c753449eaad1d76e69fffe5b'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:09.831540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-56f86665-bf02-4515-b2a5-672aacd85761 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd3dd4d7c753449eaad1d76e69fffe5b'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:09.831795 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1119/2256] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:48:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:48:09.833392 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c64fe-9151-46e6-98fc-a9067141e11e None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:09.833755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c64fe-9151-46e6-98fc-a9067141e11e None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:09.834016 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c64fe-9151-46e6-98fc-a9067141e11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:09.858533 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-389c64fe-9151-46e6-98fc-a9067141e11e None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:48:09.859729 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-389c64fe-9151-46e6-98fc-a9067141e11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:09.866133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-56f86665-bf02-4515-b2a5-672aacd85761 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd3dd4d7c753449eaad1d76e69fffe5b'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:09.872687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1120/2257] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:48:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:48:09.873711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c64fe-9151-46e6-98fc-a9067141e11e None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:09.886440 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1138/2258] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:48:09 2022] DELETE /identity/v3/users/fd3dd4d7c753449eaad1d76e69fffe5b => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:48:09.888225 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf14aaa-e351-4306-b66f-cdbb587bfa0e None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:09.888366 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c64fe-9151-46e6-98fc-a9067141e11e None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:09.888560 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf14aaa-e351-4306-b66f-cdbb587bfa0e None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:09.888799 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf14aaa-e351-4306-b66f-cdbb587bfa0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:09.925570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-ddf14aaa-e351-4306-b66f-cdbb587bfa0e None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:48:09.927258 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-ddf14aaa-e351-4306-b66f-cdbb587bfa0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:09.938217 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:48:09.953614 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a99a2-1f50-4d5c-a13c-aaa4338175ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:09.954444 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a99a2-1f50-4d5c-a13c-aaa4338175ce admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:09.954739 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a99a2-1f50-4d5c-a13c-aaa4338175ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:09.955048 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a99a2-1f50-4d5c-a13c-aaa4338175ce admin admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:09.956764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2a99a2-1f50-4d5c-a13c-aaa4338175ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:48:09.957490 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2a99a2-1f50-4d5c-a13c-aaa4338175ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:09.963428 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2a99a2-1f50-4d5c-a13c-aaa4338175ce admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:09.984968 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1121/2259] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:48:09.985922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a99a2-1f50-4d5c-a13c-aaa4338175ce admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:09.997938 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a99a2-1f50-4d5c-a13c-aaa4338175ce admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:10.003740 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe05d82-8e1b-4bd9-aaa7-6f0e1ffd608e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:10.004422 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe05d82-8e1b-4bd9-aaa7-6f0e1ffd608e admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:10.004709 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe05d82-8e1b-4bd9-aaa7-6f0e1ffd608e admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:10.005222 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe05d82-8e1b-4bd9-aaa7-6f0e1ffd608e admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:10.005935 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe05d82-8e1b-4bd9-aaa7-6f0e1ffd608e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:10.007868 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe05d82-8e1b-4bd9-aaa7-6f0e1ffd608e admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:10.053529 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1122/2260] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:48:09 2022] POST /identity/v3/projects => generated 361 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:48:10.054558 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe05d82-8e1b-4bd9-aaa7-6f0e1ffd608e admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:10.080840 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1139/2261] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:48:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:48:10.092397 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf14aaa-e351-4306-b66f-cdbb587bfa0e None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:10.093928 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-ddf14aaa-e351-4306-b66f-cdbb587bfa0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:10.143866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:48:10.144082 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe05d82-8e1b-4bd9-aaa7-6f0e1ffd608e admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:10.153544 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0bc1ed-4b22-48fd-a2a5-f6ef70a4fd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:10.154297 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0bc1ed-4b22-48fd-a2a5-f6ef70a4fd5e admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:10.154634 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0bc1ed-4b22-48fd-a2a5-f6ef70a4fd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:10.154919 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0bc1ed-4b22-48fd-a2a5-f6ef70a4fd5e admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:10.155624 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0bc1ed-4b22-48fd-a2a5-f6ef70a4fd5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:10.157758 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0bc1ed-4b22-48fd-a2a5-f6ef70a4fd5e admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:10.195048 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1123/2262] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:48:10 2022] POST /identity/v3/users => generated 281 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:48:10.202198 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0bc1ed-4b22-48fd-a2a5-f6ef70a4fd5e admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:10.214961 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf14aaa-e351-4306-b66f-cdbb587bfa0e None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:10.225208 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0bc1ed-4b22-48fd-a2a5-f6ef70a4fd5e admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:10.235314 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad958e9a-ae54-4d9a-aaf3-d7504b12f64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:10.238592 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad958e9a-ae54-4d9a-aaf3-d7504b12f64d admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:10.241383 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad958e9a-ae54-4d9a-aaf3-d7504b12f64d admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:10.242549 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad958e9a-ae54-4d9a-aaf3-d7504b12f64d admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:10.244060 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad958e9a-ae54-4d9a-aaf3-d7504b12f64d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:10.248465 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad958e9a-ae54-4d9a-aaf3-d7504b12f64d admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:10.291505 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1124/2263] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:48:10.298510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad958e9a-ae54-4d9a-aaf3-d7504b12f64d admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:10.330115 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad958e9a-ae54-4d9a-aaf3-d7504b12f64d admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:10.345711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36fdb41-9c39-4db8-9393-83315380cd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:10.349340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36fdb41-9c39-4db8-9393-83315380cd77 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:10.350366 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36fdb41-9c39-4db8-9393-83315380cd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:10.351526 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36fdb41-9c39-4db8-9393-83315380cd77 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:10.352503 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36fdb41-9c39-4db8-9393-83315380cd77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:10.354454 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36fdb41-9c39-4db8-9393-83315380cd77 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:10.357693 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce0f5a-6323-4fe6-9512-7ea8f73bd3e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:10.359103 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce0f5a-6323-4fe6-9512-7ea8f73bd3e2 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:10.359633 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce0f5a-6323-4fe6-9512-7ea8f73bd3e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:10.360403 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce0f5a-6323-4fe6-9512-7ea8f73bd3e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:10.372016 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce0f5a-6323-4fe6-9512-7ea8f73bd3e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:10.373947 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce0f5a-6323-4fe6-9512-7ea8f73bd3e2 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:10.389055 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1125/2264] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:48:10.390231 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1140/2265] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:48:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 13:48:10.393154 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36fdb41-9c39-4db8-9393-83315380cd77 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:10.407083 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36fdb41-9c39-4db8-9393-83315380cd77 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:10.418513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d435bd1-5504-4459-87f1-81defdc639be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:10.420371 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d435bd1-5504-4459-87f1-81defdc639be admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:10.420872 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d435bd1-5504-4459-87f1-81defdc639be admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:10.421398 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d435bd1-5504-4459-87f1-81defdc639be admin admin] PATH_INFO: `/v3/projects/572d04464c864f61ab6a5030f01a4496/users/4dfb450efe884bf994bcd009a3d75c96/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:10.444400 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d435bd1-5504-4459-87f1-81defdc639be admin admin] RBAC: Authorizing `identity:create_grant(project_id=572d04464c864f61ab6a5030f01a4496, user_id=4dfb450efe884bf994bcd009a3d75c96, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:10.450464 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d435bd1-5504-4459-87f1-81defdc639be admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:10.499723 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1126/2266] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:48:10 2022] PUT /identity/v3/projects/572d04464c864f61ab6a5030f01a4496/users/4dfb450efe884bf994bcd009a3d75c96/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:48:10.500880 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688f140-cfd3-40eb-8b6a-e2b76200215f None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:10.503635 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688f140-cfd3-40eb-8b6a-e2b76200215f None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:10.503930 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688f140-cfd3-40eb-8b6a-e2b76200215f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:10.595224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-1688f140-cfd3-40eb-8b6a-e2b76200215f None None] MFA Rules not processed for user `4dfb450efe884bf994bcd009a3d75c96`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:48:10.596882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-1688f140-cfd3-40eb-8b6a-e2b76200215f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:10.803662 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1141/2267] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:48:10 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:48:10.840047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d435bd1-5504-4459-87f1-81defdc639be admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:10.844498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-5d435bd1-5504-4459-87f1-81defdc639be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:10.974910 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d435bd1-5504-4459-87f1-81defdc639be admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:10.994517 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d781c036-49bb-494a-aab0-cbbf93ede040 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:10.995381 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d781c036-49bb-494a-aab0-cbbf93ede040 service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:10.995784 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d781c036-49bb-494a-aab0-cbbf93ede040 service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:10.996175 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d781c036-49bb-494a-aab0-cbbf93ede040 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:11.006613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d781c036-49bb-494a-aab0-cbbf93ede040 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:11.008362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d781c036-49bb-494a-aab0-cbbf93ede040 service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:11.020810 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1127/2268] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:48:10 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:48:11.028261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688f140-cfd3-40eb-8b6a-e2b76200215f None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:11.030362 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-1688f140-cfd3-40eb-8b6a-e2b76200215f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:11.042597 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d781c036-49bb-494a-aab0-cbbf93ede040 service cinder] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:11.061908 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:48:11.166662 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d781c036-49bb-494a-aab0-cbbf93ede040 service cinder] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:11.167023 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688f140-cfd3-40eb-8b6a-e2b76200215f None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:11.172326 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e28d8-36b0-4c60-8f79-bc2c3fd1802a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:11.172481 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51316bb9-e8b1-4810-9f87-daee4f957673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:11.173172 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e28d8-36b0-4c60-8f79-bc2c3fd1802a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:11.173455 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e28d8-36b0-4c60-8f79-bc2c3fd1802a admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:11.174081 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e28d8-36b0-4c60-8f79-bc2c3fd1802a admin admin] PATH_INFO: `/v3/projects/9eb424a779ca421c95184a3a189b558f` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:11.174247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51316bb9-e8b1-4810-9f87-daee4f957673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:11.174642 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51316bb9-e8b1-4810-9f87-daee4f957673 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:11.175067 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51316bb9-e8b1-4810-9f87-daee4f957673 admin admin] PATH_INFO: `/v3/users/4dfb450efe884bf994bcd009a3d75c96` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:11.175187 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e28d8-36b0-4c60-8f79-bc2c3fd1802a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eb424a779ca421c95184a3a189b558f)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:11.176666 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51316bb9-e8b1-4810-9f87-daee4f957673 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dfb450efe884bf994bcd009a3d75c96)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:11.177556 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e28d8-36b0-4c60-8f79-bc2c3fd1802a admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:11.180493 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51316bb9-e8b1-4810-9f87-daee4f957673 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:11.191708 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:48:11.279627 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-51316bb9-e8b1-4810-9f87-daee4f957673 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dfb450efe884bf994bcd009a3d75c96'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:11.294653 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-7a4e28d8-36b0-4c60-8f79-bc2c3fd1802a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eb424a779ca421c95184a3a189b558f', 'request_id': 'req-7a4e28d8-36b0-4c60-8f79-bc2c3fd1802a'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:11.295582 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-51316bb9-e8b1-4810-9f87-daee4f957673 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dfb450efe884bf994bcd009a3d75c96'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:11.299334 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-51316bb9-e8b1-4810-9f87-daee4f957673 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dfb450efe884bf994bcd009a3d75c96'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:11.307432 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-7a4e28d8-36b0-4c60-8f79-bc2c3fd1802a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eb424a779ca421c95184a3a189b558f', 'request_id': 'req-7a4e28d8-36b0-4c60-8f79-bc2c3fd1802a'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:11.330726 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-51316bb9-e8b1-4810-9f87-daee4f957673 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dfb450efe884bf994bcd009a3d75c96'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:11.331785 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-7a4e28d8-36b0-4c60-8f79-bc2c3fd1802a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eb424a779ca421c95184a3a189b558f', 'request_id': 'req-7a4e28d8-36b0-4c60-8f79-bc2c3fd1802a'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:11.366647 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-7a4e28d8-36b0-4c60-8f79-bc2c3fd1802a admin admin] The token cache is being invalidate because project 9eb424a779ca421c95184a3a189b558f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:48:11.367047 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-7a4e28d8-36b0-4c60-8f79-bc2c3fd1802a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:11.367717 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-7a4e28d8-36b0-4c60-8f79-bc2c3fd1802a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:11.372327 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1142/2269] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:48:11 2022] DELETE /identity/v3/projects/9eb424a779ca421c95184a3a189b558f => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:48:11.374018 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e28d8-36b0-4c60-8f79-bc2c3fd1802a admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:11.375328 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-7a4e28d8-36b0-4c60-8f79-bc2c3fd1802a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:11.387514 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-51316bb9-e8b1-4810-9f87-daee4f957673 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dfb450efe884bf994bcd009a3d75c96'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:11.405652 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1128/2270] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:48:11 2022] DELETE /identity/v3/users/4dfb450efe884bf994bcd009a3d75c96 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:48:11.411743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd34bb-2313-48c4-8b08-29ec7525f885 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:11.411743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd34bb-2313-48c4-8b08-29ec7525f885 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:11.412009 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd34bb-2313-48c4-8b08-29ec7525f885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:11.446525 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-5bcd34bb-2313-48c4-8b08-29ec7525f885 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:48:11.449645 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-5bcd34bb-2313-48c4-8b08-29ec7525f885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:11.459071 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e28d8-36b0-4c60-8f79-bc2c3fd1802a admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:11.535313 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1129/2271] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:48:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:48:11.542522 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa2b20-ed46-4510-ada6-bc7cbf2851ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:11.543615 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa2b20-ed46-4510-ada6-bc7cbf2851ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:11.543911 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa2b20-ed46-4510-ada6-bc7cbf2851ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:11.544268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa2b20-ed46-4510-ada6-bc7cbf2851ea admin admin] PATH_INFO: `/v3/users/5cc2da1f46d0407fbda259a694f5d81f` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:11.545993 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fa2b20-ed46-4510-ada6-bc7cbf2851ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cc2da1f46d0407fbda259a694f5d81f)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:11.546215 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd34bb-2313-48c4-8b08-29ec7525f885 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:11.547795 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-5bcd34bb-2313-48c4-8b08-29ec7525f885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:11.547917 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fa2b20-ed46-4510-ada6-bc7cbf2851ea admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:11.585759 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:48:11.627578 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd34bb-2313-48c4-8b08-29ec7525f885 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:11.627838 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-61fa2b20-ed46-4510-ada6-bc7cbf2851ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cc2da1f46d0407fbda259a694f5d81f'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:11.631112 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:48:11.633628 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257902b-5fcc-47f1-893a-b5f9720669ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:11.634449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257902b-5fcc-47f1-893a-b5f9720669ce service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:11.634847 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257902b-5fcc-47f1-893a-b5f9720669ce service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:11.635428 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257902b-5fcc-47f1-893a-b5f9720669ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:11.642035 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-61fa2b20-ed46-4510-ada6-bc7cbf2851ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cc2da1f46d0407fbda259a694f5d81f'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:11.649604 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b257902b-5fcc-47f1-893a-b5f9720669ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:11.651354 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b257902b-5fcc-47f1-893a-b5f9720669ce service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:11.652920 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-61fa2b20-ed46-4510-ada6-bc7cbf2851ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cc2da1f46d0407fbda259a694f5d81f'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:11.653808 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-61fa2b20-ed46-4510-ada6-bc7cbf2851ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cc2da1f46d0407fbda259a694f5d81f'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:11.663377 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1130/2272] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:48:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 13:48:11.685650 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-61fa2b20-ed46-4510-ada6-bc7cbf2851ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cc2da1f46d0407fbda259a694f5d81f'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:11.699673 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1143/2273] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:48:11 2022] DELETE /identity/v3/users/5cc2da1f46d0407fbda259a694f5d81f => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:48:12.173039 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257902b-5fcc-47f1-893a-b5f9720669ce service neutron] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:12.175254 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-b257902b-5fcc-47f1-893a-b5f9720669ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:12.189661 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257902b-5fcc-47f1-893a-b5f9720669ce service neutron] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:12.241137 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa2b20-ed46-4510-ada6-bc7cbf2851ea admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:12.243192 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-61fa2b20-ed46-4510-ada6-bc7cbf2851ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:12.315548 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12feaf-cb38-4385-8979-3c3424b44b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:12.316340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12feaf-cb38-4385-8979-3c3424b44b99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:12.316687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12feaf-cb38-4385-8979-3c3424b44b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:12.317094 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12feaf-cb38-4385-8979-3c3424b44b99 admin admin] PATH_INFO: `/v3/projects/572d04464c864f61ab6a5030f01a4496` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:12.318728 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc12feaf-cb38-4385-8979-3c3424b44b99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=572d04464c864f61ab6a5030f01a4496)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:12.321765 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc12feaf-cb38-4385-8979-3c3424b44b99 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:12.372632 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa2b20-ed46-4510-ada6-bc7cbf2851ea admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:12.377671 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23f44f-eb0f-44e0-8a2a-607a993a0ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:12.378533 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23f44f-eb0f-44e0-8a2a-607a993a0ad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:12.378817 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23f44f-eb0f-44e0-8a2a-607a993a0ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:12.379096 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23f44f-eb0f-44e0-8a2a-607a993a0ad1 admin admin] PATH_INFO: `/v3/projects/c32dd3295f4445669e4ec1cccbea13e2` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:12.380434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23f44f-eb0f-44e0-8a2a-607a993a0ad1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c32dd3295f4445669e4ec1cccbea13e2)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:12.384707 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23f44f-eb0f-44e0-8a2a-607a993a0ad1 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:12.404592 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-bc12feaf-cb38-4385-8979-3c3424b44b99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '572d04464c864f61ab6a5030f01a4496', 'request_id': 'req-bc12feaf-cb38-4385-8979-3c3424b44b99'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:12.412626 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-bc12feaf-cb38-4385-8979-3c3424b44b99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '572d04464c864f61ab6a5030f01a4496', 'request_id': 'req-bc12feaf-cb38-4385-8979-3c3424b44b99'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:12.427745 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-bc12feaf-cb38-4385-8979-3c3424b44b99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '572d04464c864f61ab6a5030f01a4496', 'request_id': 'req-bc12feaf-cb38-4385-8979-3c3424b44b99'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:12.445857 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-bc12feaf-cb38-4385-8979-3c3424b44b99 admin admin] The token cache is being invalidate because project 572d04464c864f61ab6a5030f01a4496 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:48:12.446430 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-bc12feaf-cb38-4385-8979-3c3424b44b99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:12.447691 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-bc12feaf-cb38-4385-8979-3c3424b44b99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:12.450105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-fe23f44f-eb0f-44e0-8a2a-607a993a0ad1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c32dd3295f4445669e4ec1cccbea13e2', 'request_id': 'req-fe23f44f-eb0f-44e0-8a2a-607a993a0ad1'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:12.469035 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1131/2274] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:48:12 2022] DELETE /identity/v3/projects/572d04464c864f61ab6a5030f01a4496 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:48:12.487841 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-fe23f44f-eb0f-44e0-8a2a-607a993a0ad1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c32dd3295f4445669e4ec1cccbea13e2', 'request_id': 'req-fe23f44f-eb0f-44e0-8a2a-607a993a0ad1'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:12.506099 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-fe23f44f-eb0f-44e0-8a2a-607a993a0ad1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c32dd3295f4445669e4ec1cccbea13e2', 'request_id': 'req-fe23f44f-eb0f-44e0-8a2a-607a993a0ad1'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:12.523326 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-fe23f44f-eb0f-44e0-8a2a-607a993a0ad1 admin admin] The token cache is being invalidate because project c32dd3295f4445669e4ec1cccbea13e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:48:12.524123 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-fe23f44f-eb0f-44e0-8a2a-607a993a0ad1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:12.525158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-fe23f44f-eb0f-44e0-8a2a-607a993a0ad1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:12.527532 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1144/2275] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:48:12 2022] DELETE /identity/v3/projects/c32dd3295f4445669e4ec1cccbea13e2 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:48:12.532226 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a134840d-8498-46f8-b2b8-b35e9292dd40 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:12.533198 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a134840d-8498-46f8-b2b8-b35e9292dd40 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:12.534776 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a134840d-8498-46f8-b2b8-b35e9292dd40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:12.568271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-a134840d-8498-46f8-b2b8-b35e9292dd40 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:48:12.569901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a134840d-8498-46f8-b2b8-b35e9292dd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:12.649585 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:48:12.663833 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1132/2276] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:48:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:48:12.681896 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb8297-f747-40a7-ae0c-b1aa81dfe121 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:12.682226 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb8297-f747-40a7-ae0c-b1aa81dfe121 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:12.682461 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb8297-f747-40a7-ae0c-b1aa81dfe121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:12.729258 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-8edb8297-f747-40a7-ae0c-b1aa81dfe121 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:48:12.729258 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-8edb8297-f747-40a7-ae0c-b1aa81dfe121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:12.734990 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1145/2277] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:48:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:48:12.739084 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a134840d-8498-46f8-b2b8-b35e9292dd40 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:12.749156 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a134840d-8498-46f8-b2b8-b35e9292dd40 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:12.754226 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6432442c-b231-4f89-92ec-70f4765c3818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:12.754897 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432442c-b231-4f89-92ec-70f4765c3818 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:12.755180 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432442c-b231-4f89-92ec-70f4765c3818 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:12.755455 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432442c-b231-4f89-92ec-70f4765c3818 admin admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:12.756191 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6432442c-b231-4f89-92ec-70f4765c3818 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:48:12.756498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6432442c-b231-4f89-92ec-70f4765c3818 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:12.758467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6432442c-b231-4f89-92ec-70f4765c3818 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:12.772072 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1133/2278] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:48:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:48:12.776393 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb8297-f747-40a7-ae0c-b1aa81dfe121 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:12.787863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb8297-f747-40a7-ae0c-b1aa81dfe121 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:12.793328 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60183e63-fba2-4022-9ba0-93d657c577a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:12.793993 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60183e63-fba2-4022-9ba0-93d657c577a4 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:12.794263 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60183e63-fba2-4022-9ba0-93d657c577a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:12.794537 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60183e63-fba2-4022-9ba0-93d657c577a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:12.795232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60183e63-fba2-4022-9ba0-93d657c577a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:12.797133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60183e63-fba2-4022-9ba0-93d657c577a4 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:12.830647 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1146/2279] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:48:12 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:48:12.833607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6432442c-b231-4f89-92ec-70f4765c3818 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:12.844762 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6432442c-b231-4f89-92ec-70f4765c3818 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:12.917325 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f28bd-f5f2-4713-877e-0f136fe41341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:12.918117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f28bd-f5f2-4713-877e-0f136fe41341 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:12.918486 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f28bd-f5f2-4713-877e-0f136fe41341 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:12.918849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f28bd-f5f2-4713-877e-0f136fe41341 admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:12.919650 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f28bd-f5f2-4713-877e-0f136fe41341 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:12.921703 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f28bd-f5f2-4713-877e-0f136fe41341 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:12.946946 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1134/2280] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:48:12 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:48:12.952016 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60183e63-fba2-4022-9ba0-93d657c577a4 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:12.968951 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60183e63-fba2-4022-9ba0-93d657c577a4 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:12.975195 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c459cc-0cd0-429b-b175-e80536a1a1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:12.976005 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c459cc-0cd0-429b-b175-e80536a1a1ec admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:12.976305 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c459cc-0cd0-429b-b175-e80536a1a1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:12.976589 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c459cc-0cd0-429b-b175-e80536a1a1ec admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:12.977392 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c459cc-0cd0-429b-b175-e80536a1a1ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:12.979015 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c459cc-0cd0-429b-b175-e80536a1a1ec admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:12.993422 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1147/2281] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:48:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:48:12.994847 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f28bd-f5f2-4713-877e-0f136fe41341 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:13.005878 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f28bd-f5f2-4713-877e-0f136fe41341 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:13.010605 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa3deb-b4c8-4c5d-a003-f43ae5c497a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:13.011261 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa3deb-b4c8-4c5d-a003-f43ae5c497a6 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:13.011547 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa3deb-b4c8-4c5d-a003-f43ae5c497a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:13.011824 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa3deb-b4c8-4c5d-a003-f43ae5c497a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:13.012534 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afa3deb-b4c8-4c5d-a003-f43ae5c497a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:13.014161 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afa3deb-b4c8-4c5d-a003-f43ae5c497a6 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:13.028775 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1135/2282] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:48:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:48:13.030628 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c459cc-0cd0-429b-b175-e80536a1a1ec admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:13.043256 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c459cc-0cd0-429b-b175-e80536a1a1ec admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:13.048774 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46602bcb-64d5-4a43-8e5b-46761dec4ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:13.050529 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46602bcb-64d5-4a43-8e5b-46761dec4ca6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:13.051268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46602bcb-64d5-4a43-8e5b-46761dec4ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:13.051608 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46602bcb-64d5-4a43-8e5b-46761dec4ca6 admin admin] PATH_INFO: `/v3/projects/a07d022b86444e2d8cffff83e458e20f/users/28c4ef1360cb4069b0259459eed396c1/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:13.066122 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:INSERT=3,keystone:DELETE=10,keystone:SELECT=87 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:48:13.070090 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46602bcb-64d5-4a43-8e5b-46761dec4ca6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a07d022b86444e2d8cffff83e458e20f, user_id=28c4ef1360cb4069b0259459eed396c1, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:13.074803 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46602bcb-64d5-4a43-8e5b-46761dec4ca6 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:13.120698 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1148/2283] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:48:13 2022] PUT /identity/v3/projects/a07d022b86444e2d8cffff83e458e20f/users/28c4ef1360cb4069b0259459eed396c1/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:48:13.124879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54497455-f79c-4c7c-90d1-7a703ff2895f None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:13.125513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54497455-f79c-4c7c-90d1-7a703ff2895f None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:13.125961 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54497455-f79c-4c7c-90d1-7a703ff2895f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:13.166922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-54497455-f79c-4c7c-90d1-7a703ff2895f None None] MFA Rules not processed for user `28c4ef1360cb4069b0259459eed396c1`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:48:13.167945 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-54497455-f79c-4c7c-90d1-7a703ff2895f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:13.342441 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1136/2284] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:48:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:48:13.347697 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46602bcb-64d5-4a43-8e5b-46761dec4ca6 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:13.478854 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46602bcb-64d5-4a43-8e5b-46761dec4ca6 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:13.490322 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da0c133-b502-4d05-ac44-7f99d55d02c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:13.491055 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0c133-b502-4d05-ac44-7f99d55d02c9 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:13.491341 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0c133-b502-4d05-ac44-7f99d55d02c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:13.491619 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0c133-b502-4d05-ac44-7f99d55d02c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:13.492327 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da0c133-b502-4d05-ac44-7f99d55d02c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:13.494270 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da0c133-b502-4d05-ac44-7f99d55d02c9 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:13.569598 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54497455-f79c-4c7c-90d1-7a703ff2895f None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:13.576702 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1149/2285] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:48:13 2022] POST /identity/v3/projects => generated 385 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:48:13.586044 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54497455-f79c-4c7c-90d1-7a703ff2895f None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:13.667850 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:48:13.686395 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b9809c-7dd1-4dab-a281-ef2f2e5ff267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:13.687093 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9809c-7dd1-4dab-a281-ef2f2e5ff267 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:13.687483 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9809c-7dd1-4dab-a281-ef2f2e5ff267 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:13.687907 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9809c-7dd1-4dab-a281-ef2f2e5ff267 admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:13.688615 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b9809c-7dd1-4dab-a281-ef2f2e5ff267 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:13.691458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b9809c-7dd1-4dab-a281-ef2f2e5ff267 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:13.729029 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da0c133-b502-4d05-ac44-7f99d55d02c9 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:13.731242 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1137/2286] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:48:13 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:48:13.743246 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da0c133-b502-4d05-ac44-7f99d55d02c9 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:13.757650 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040159ab-0319-4085-9470-b04ae729fd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:13.758432 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040159ab-0319-4085-9470-b04ae729fd92 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:13.758818 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040159ab-0319-4085-9470-b04ae729fd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:13.759213 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040159ab-0319-4085-9470-b04ae729fd92 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:13.760862 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040159ab-0319-4085-9470-b04ae729fd92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:13.770212 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040159ab-0319-4085-9470-b04ae729fd92 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:13.803473 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b9809c-7dd1-4dab-a281-ef2f2e5ff267 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:13.812535 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1150/2287] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:48:13.820829 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b9809c-7dd1-4dab-a281-ef2f2e5ff267 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:13.837754 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770b600-a7bb-4e38-a4a5-ebf1c978b713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:13.838449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770b600-a7bb-4e38-a4a5-ebf1c978b713 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:13.838813 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770b600-a7bb-4e38-a4a5-ebf1c978b713 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:13.839104 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770b600-a7bb-4e38-a4a5-ebf1c978b713 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:13.839909 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770b600-a7bb-4e38-a4a5-ebf1c978b713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:13.842871 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770b600-a7bb-4e38-a4a5-ebf1c978b713 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:13.868710 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040159ab-0319-4085-9470-b04ae729fd92 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:13.876168 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1138/2288] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:48:13.880786 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040159ab-0319-4085-9470-b04ae729fd92 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:13.898095 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4856d6e-aef9-471e-b421-43317af272da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:13.900125 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4856d6e-aef9-471e-b421-43317af272da admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:13.900938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4856d6e-aef9-471e-b421-43317af272da admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:13.901620 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4856d6e-aef9-471e-b421-43317af272da admin admin] PATH_INFO: `/v3/projects/8da18d18245b46ef84805364b8a46227/users/f07264b84c334dbb96bc2c98d2bf75ba/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:13.919188 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4856d6e-aef9-471e-b421-43317af272da admin admin] RBAC: Authorizing `identity:create_grant(project_id=8da18d18245b46ef84805364b8a46227, user_id=f07264b84c334dbb96bc2c98d2bf75ba, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:13.922403 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4856d6e-aef9-471e-b421-43317af272da admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:13.945687 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1151/2289] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:48:13 2022] PUT /identity/v3/projects/8da18d18245b46ef84805364b8a46227/users/f07264b84c334dbb96bc2c98d2bf75ba/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:48:13.960445 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524ddb0-91ba-4a93-a408-0c6505740b18 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:13.960845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524ddb0-91ba-4a93-a408-0c6505740b18 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:13.961252 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524ddb0-91ba-4a93-a408-0c6505740b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:13.998579 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-3524ddb0-91ba-4a93-a408-0c6505740b18 None None] MFA Rules not processed for user `f07264b84c334dbb96bc2c98d2bf75ba`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:48:14.000282 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-3524ddb0-91ba-4a93-a408-0c6505740b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:14.096641 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1139/2290] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:48:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:48:14.097680 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4856d6e-aef9-471e-b421-43317af272da admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:14.175435 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4856d6e-aef9-471e-b421-43317af272da admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:14.180956 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa191b7-65b0-4b90-a758-4ac901f57646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:14.181659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa191b7-65b0-4b90-a758-4ac901f57646 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:14.182081 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa191b7-65b0-4b90-a758-4ac901f57646 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:14.182367 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa191b7-65b0-4b90-a758-4ac901f57646 admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:14.183086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa191b7-65b0-4b90-a758-4ac901f57646 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:14.185019 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa191b7-65b0-4b90-a758-4ac901f57646 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:14.240632 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1152/2291] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:48:14 2022] POST /identity/v3/projects => generated 383 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:48:14.240873 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524ddb0-91ba-4a93-a408-0c6505740b18 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:14.273992 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524ddb0-91ba-4a93-a408-0c6505740b18 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:14.430334 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f00c7-3921-445f-ad5e-0efc3d174119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:14.431306 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f00c7-3921-445f-ad5e-0efc3d174119 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:14.431865 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f00c7-3921-445f-ad5e-0efc3d174119 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:14.432297 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f00c7-3921-445f-ad5e-0efc3d174119 admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:14.433333 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538f00c7-3921-445f-ad5e-0efc3d174119 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:14.435645 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538f00c7-3921-445f-ad5e-0efc3d174119 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:14.465072 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1140/2292] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:48:14 2022] POST /identity/v3/users => generated 298 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:48:14.470172 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa191b7-65b0-4b90-a758-4ac901f57646 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:14.488213 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa191b7-65b0-4b90-a758-4ac901f57646 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:14.495862 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e267c44-d977-4f9b-b407-820bed6dc5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:14.496175 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e267c44-d977-4f9b-b407-820bed6dc5d0 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:14.496837 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e267c44-d977-4f9b-b407-820bed6dc5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:14.497158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e267c44-d977-4f9b-b407-820bed6dc5d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:14.497911 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e267c44-d977-4f9b-b407-820bed6dc5d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:14.499588 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e267c44-d977-4f9b-b407-820bed6dc5d0 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:14.528089 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1153/2293] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:48:14.534219 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f00c7-3921-445f-ad5e-0efc3d174119 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:14.547280 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f00c7-3921-445f-ad5e-0efc3d174119 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:14.555694 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962303c3-1ea2-4683-8704-75b7041223e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:14.556607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962303c3-1ea2-4683-8704-75b7041223e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:14.556890 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962303c3-1ea2-4683-8704-75b7041223e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:14.557204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962303c3-1ea2-4683-8704-75b7041223e8 admin admin] PATH_INFO: `/v3/projects/09128e2092e944b88023fc4a7c9d8411/users/77b13775ee864057801505a005944418/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:14.575566 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962303c3-1ea2-4683-8704-75b7041223e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09128e2092e944b88023fc4a7c9d8411, user_id=77b13775ee864057801505a005944418, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:14.578897 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962303c3-1ea2-4683-8704-75b7041223e8 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:14.597193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1141/2294] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:48:14 2022] PUT /identity/v3/projects/09128e2092e944b88023fc4a7c9d8411/users/77b13775ee864057801505a005944418/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:48:14.601247 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e267c44-d977-4f9b-b407-820bed6dc5d0 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:14.620135 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e267c44-d977-4f9b-b407-820bed6dc5d0 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:14.643856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:48:14.704304 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30e8a8-9048-4542-b0d0-afd5f23f2b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:14.705740 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30e8a8-9048-4542-b0d0-afd5f23f2b36 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:14.706067 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30e8a8-9048-4542-b0d0-afd5f23f2b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:14.706358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30e8a8-9048-4542-b0d0-afd5f23f2b36 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:14.707131 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30e8a8-9048-4542-b0d0-afd5f23f2b36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:14.709072 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30e8a8-9048-4542-b0d0-afd5f23f2b36 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:14.719862 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1154/2295] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:48:14.724124 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962303c3-1ea2-4683-8704-75b7041223e8 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:14.735967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962303c3-1ea2-4683-8704-75b7041223e8 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:14.741376 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e621ae12-d07a-46df-b890-f9ffdc2327b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:14.742399 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621ae12-d07a-46df-b890-f9ffdc2327b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:14.742687 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621ae12-d07a-46df-b890-f9ffdc2327b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:14.742966 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621ae12-d07a-46df-b890-f9ffdc2327b7 admin admin] PATH_INFO: `/v3/projects/09128e2092e944b88023fc4a7c9d8411/users/77b13775ee864057801505a005944418/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:14.745223 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e621ae12-d07a-46df-b890-f9ffdc2327b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09128e2092e944b88023fc4a7c9d8411, user_id=77b13775ee864057801505a005944418, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:14.748562 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e621ae12-d07a-46df-b890-f9ffdc2327b7 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:14.770970 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1142/2296] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:48:14 2022] PUT /identity/v3/projects/09128e2092e944b88023fc4a7c9d8411/users/77b13775ee864057801505a005944418/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:48:14.772808 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618cc1c-30e2-4bfe-bbcb-578622966310 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:14.773295 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618cc1c-30e2-4bfe-bbcb-578622966310 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:14.773587 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618cc1c-30e2-4bfe-bbcb-578622966310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:14.807819 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-5618cc1c-30e2-4bfe-bbcb-578622966310 None None] MFA Rules not processed for user `77b13775ee864057801505a005944418`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:48:14.810910 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-5618cc1c-30e2-4bfe-bbcb-578622966310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:14.902016 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1155/2297] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:48:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:48:14.915838 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e621ae12-d07a-46df-b890-f9ffdc2327b7 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:14.917738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-e621ae12-d07a-46df-b890-f9ffdc2327b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:14.939353 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:48:15.011731 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e621ae12-d07a-46df-b890-f9ffdc2327b7 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:15.015735 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c28f4da-4864-4668-8b4f-b2be0021bbb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:15.016537 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28f4da-4864-4668-8b4f-b2be0021bbb6 service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:15.016826 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28f4da-4864-4668-8b4f-b2be0021bbb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:15.017164 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28f4da-4864-4668-8b4f-b2be0021bbb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:15.100462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c28f4da-4864-4668-8b4f-b2be0021bbb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:15.102465 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c28f4da-4864-4668-8b4f-b2be0021bbb6 service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:15.128291 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1143/2298] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:48:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:48:16.822298 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618cc1c-30e2-4bfe-bbcb-578622966310 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:16.824122 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-5618cc1c-30e2-4bfe-bbcb-578622966310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:16.842643 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618cc1c-30e2-4bfe-bbcb-578622966310 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:16.848655 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879fa851-df44-4148-8463-c2371154e831 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:16.849658 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fa851-df44-4148-8463-c2371154e831 service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:16.850188 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fa851-df44-4148-8463-c2371154e831 service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:16.850973 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fa851-df44-4148-8463-c2371154e831 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:16.944576 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879fa851-df44-4148-8463-c2371154e831 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:16.946614 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879fa851-df44-4148-8463-c2371154e831 service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:16.957902 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1156/2299] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:48:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:48:17.660613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c28f4da-4864-4668-8b4f-b2be0021bbb6 service cinder] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:17.672976 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c28f4da-4864-4668-8b4f-b2be0021bbb6 service cinder] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:17.677710 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c6751-7613-436e-af87-da2425c3a4e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:17.678503 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c6751-7613-436e-af87-da2425c3a4e0 service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:17.678793 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c6751-7613-436e-af87-da2425c3a4e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:17.679097 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c6751-7613-436e-af87-da2425c3a4e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:17.691554 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c6751-7613-436e-af87-da2425c3a4e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:17.692993 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c6751-7613-436e-af87-da2425c3a4e0 service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:17.703518 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1144/2300] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:48:17 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:48:21.798981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879fa851-df44-4148-8463-c2371154e831 service cinder] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:21.852981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:48:21.881766 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879fa851-df44-4148-8463-c2371154e831 service cinder] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:21.890383 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06fbfee-90cc-4a36-9bc2-d5b09b125d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:21.891267 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06fbfee-90cc-4a36-9bc2-d5b09b125d06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:21.891590 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06fbfee-90cc-4a36-9bc2-d5b09b125d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:21.891870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06fbfee-90cc-4a36-9bc2-d5b09b125d06 admin admin] PATH_INFO: `/v3/users/28c4ef1360cb4069b0259459eed396c1` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:21.893832 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06fbfee-90cc-4a36-9bc2-d5b09b125d06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28c4ef1360cb4069b0259459eed396c1)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:21.895797 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06fbfee-90cc-4a36-9bc2-d5b09b125d06 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:21.957358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-f06fbfee-90cc-4a36-9bc2-d5b09b125d06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28c4ef1360cb4069b0259459eed396c1'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:21.965999 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-f06fbfee-90cc-4a36-9bc2-d5b09b125d06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28c4ef1360cb4069b0259459eed396c1'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:21.974118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-f06fbfee-90cc-4a36-9bc2-d5b09b125d06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28c4ef1360cb4069b0259459eed396c1'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:21.974793 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-f06fbfee-90cc-4a36-9bc2-d5b09b125d06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28c4ef1360cb4069b0259459eed396c1'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:21.995029 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-f06fbfee-90cc-4a36-9bc2-d5b09b125d06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28c4ef1360cb4069b0259459eed396c1'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:22.007355 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1157/2301] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:48:21 2022] DELETE /identity/v3/users/28c4ef1360cb4069b0259459eed396c1 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:48:22.011781 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd66967-8d92-4b31-b43a-f88704e8808e None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:22.012323 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd66967-8d92-4b31-b43a-f88704e8808e None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:22.012323 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd66967-8d92-4b31-b43a-f88704e8808e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:22.042314 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-8bd66967-8d92-4b31-b43a-f88704e8808e None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:48:22.044452 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-8bd66967-8d92-4b31-b43a-f88704e8808e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:22.171696 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1145/2302] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:48:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:48:22.183234 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06fbfee-90cc-4a36-9bc2-d5b09b125d06 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:22.185241 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-f06fbfee-90cc-4a36-9bc2-d5b09b125d06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:22.340167 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06fbfee-90cc-4a36-9bc2-d5b09b125d06 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:22.351836 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f32b5-c52c-4e87-9d47-b603620439d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:22.353212 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f32b5-c52c-4e87-9d47-b603620439d4 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:22.353731 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f32b5-c52c-4e87-9d47-b603620439d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:22.354119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f32b5-c52c-4e87-9d47-b603620439d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:22.383707 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f32b5-c52c-4e87-9d47-b603620439d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:22.387069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f32b5-c52c-4e87-9d47-b603620439d4 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:22.404445 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1158/2303] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:48:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 13:48:22.856188 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd66967-8d92-4b31-b43a-f88704e8808e None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:22.857714 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-8bd66967-8d92-4b31-b43a-f88704e8808e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:22.888903 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd66967-8d92-4b31-b43a-f88704e8808e None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:22.895446 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dbc17d-9302-4ff3-b227-40ce3519f231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:22.896517 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dbc17d-9302-4ff3-b227-40ce3519f231 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:22.897169 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dbc17d-9302-4ff3-b227-40ce3519f231 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:22.897693 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dbc17d-9302-4ff3-b227-40ce3519f231 admin admin] PATH_INFO: `/v3/projects/a07d022b86444e2d8cffff83e458e20f` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:22.899985 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dbc17d-9302-4ff3-b227-40ce3519f231 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a07d022b86444e2d8cffff83e458e20f)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:22.904280 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dbc17d-9302-4ff3-b227-40ce3519f231 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:22.955132 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:48:22.974918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-67dbc17d-9302-4ff3-b227-40ce3519f231 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a07d022b86444e2d8cffff83e458e20f', 'request_id': 'req-67dbc17d-9302-4ff3-b227-40ce3519f231'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:22.981866 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-67dbc17d-9302-4ff3-b227-40ce3519f231 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a07d022b86444e2d8cffff83e458e20f', 'request_id': 'req-67dbc17d-9302-4ff3-b227-40ce3519f231'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:22.993712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-67dbc17d-9302-4ff3-b227-40ce3519f231 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a07d022b86444e2d8cffff83e458e20f', 'request_id': 'req-67dbc17d-9302-4ff3-b227-40ce3519f231'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:23.013387 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-67dbc17d-9302-4ff3-b227-40ce3519f231 admin admin] The token cache is being invalidate because project a07d022b86444e2d8cffff83e458e20f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:48:23.014116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-67dbc17d-9302-4ff3-b227-40ce3519f231 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:23.014471 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-67dbc17d-9302-4ff3-b227-40ce3519f231 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:23.016063 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1146/2304] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:48:22 2022] DELETE /identity/v3/projects/a07d022b86444e2d8cffff83e458e20f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:48:23.020814 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f32b5-c52c-4e87-9d47-b603620439d4 service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:23.030847 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f32b5-c52c-4e87-9d47-b603620439d4 service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:23.124688 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:48:23.133821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6eeff7-0dbf-4981-8a3c-e251a39858c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:23.134894 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6eeff7-0dbf-4981-8a3c-e251a39858c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:23.135271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6eeff7-0dbf-4981-8a3c-e251a39858c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:23.135658 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6eeff7-0dbf-4981-8a3c-e251a39858c5 admin admin] PATH_INFO: `/v3/users/f07264b84c334dbb96bc2c98d2bf75ba` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:23.137839 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6eeff7-0dbf-4981-8a3c-e251a39858c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f07264b84c334dbb96bc2c98d2bf75ba)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:23.144363 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6eeff7-0dbf-4981-8a3c-e251a39858c5 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:23.227242 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-bf6eeff7-0dbf-4981-8a3c-e251a39858c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f07264b84c334dbb96bc2c98d2bf75ba'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:23.242468 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-bf6eeff7-0dbf-4981-8a3c-e251a39858c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f07264b84c334dbb96bc2c98d2bf75ba'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:23.254172 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-bf6eeff7-0dbf-4981-8a3c-e251a39858c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f07264b84c334dbb96bc2c98d2bf75ba'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:23.254904 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-bf6eeff7-0dbf-4981-8a3c-e251a39858c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f07264b84c334dbb96bc2c98d2bf75ba'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:23.272403 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-bf6eeff7-0dbf-4981-8a3c-e251a39858c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f07264b84c334dbb96bc2c98d2bf75ba'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:23.283632 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1159/2305] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:48:23 2022] DELETE /identity/v3/users/f07264b84c334dbb96bc2c98d2bf75ba => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:48:23.817888 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dbc17d-9302-4ff3-b227-40ce3519f231 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:23.821063 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-67dbc17d-9302-4ff3-b227-40ce3519f231 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:23.856369 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dbc17d-9302-4ff3-b227-40ce3519f231 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:23.988000 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e8d86-2f61-4756-b23d-a208b6e2b192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:23.989658 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e8d86-2f61-4756-b23d-a208b6e2b192 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:23.990200 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e8d86-2f61-4756-b23d-a208b6e2b192 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:23.990630 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e8d86-2f61-4756-b23d-a208b6e2b192 admin admin] PATH_INFO: `/v3/projects/8da18d18245b46ef84805364b8a46227` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:23.992055 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88e8d86-2f61-4756-b23d-a208b6e2b192 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8da18d18245b46ef84805364b8a46227)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:23.994957 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88e8d86-2f61-4756-b23d-a208b6e2b192 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:24.060270 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-d88e8d86-2f61-4756-b23d-a208b6e2b192 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8da18d18245b46ef84805364b8a46227', 'request_id': 'req-d88e8d86-2f61-4756-b23d-a208b6e2b192'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:24.067946 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-d88e8d86-2f61-4756-b23d-a208b6e2b192 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8da18d18245b46ef84805364b8a46227', 'request_id': 'req-d88e8d86-2f61-4756-b23d-a208b6e2b192'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:24.083627 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-d88e8d86-2f61-4756-b23d-a208b6e2b192 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8da18d18245b46ef84805364b8a46227', 'request_id': 'req-d88e8d86-2f61-4756-b23d-a208b6e2b192'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:24.103044 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-d88e8d86-2f61-4756-b23d-a208b6e2b192 admin admin] The token cache is being invalidate because project 8da18d18245b46ef84805364b8a46227 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:48:24.103776 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-d88e8d86-2f61-4756-b23d-a208b6e2b192 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:24.104645 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-d88e8d86-2f61-4756-b23d-a208b6e2b192 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:24.109147 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1147/2306] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:48:23 2022] DELETE /identity/v3/projects/8da18d18245b46ef84805364b8a46227 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:48:24.113876 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6eeff7-0dbf-4981-8a3c-e251a39858c5 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:24.115771 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-bf6eeff7-0dbf-4981-8a3c-e251a39858c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:24.132375 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6eeff7-0dbf-4981-8a3c-e251a39858c5 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:24.236260 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a33c5ef-f86f-407f-8b4d-5fb9f1d797c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:24.238084 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33c5ef-f86f-407f-8b4d-5fb9f1d797c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:24.238552 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33c5ef-f86f-407f-8b4d-5fb9f1d797c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:24.238990 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33c5ef-f86f-407f-8b4d-5fb9f1d797c1 admin admin] PATH_INFO: `/v3/users/77b13775ee864057801505a005944418` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:24.241161 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a33c5ef-f86f-407f-8b4d-5fb9f1d797c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77b13775ee864057801505a005944418)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:24.243467 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a33c5ef-f86f-407f-8b4d-5fb9f1d797c1 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:24.318897 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8a33c5ef-f86f-407f-8b4d-5fb9f1d797c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77b13775ee864057801505a005944418'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:24.330983 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8a33c5ef-f86f-407f-8b4d-5fb9f1d797c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77b13775ee864057801505a005944418'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:24.340650 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8a33c5ef-f86f-407f-8b4d-5fb9f1d797c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77b13775ee864057801505a005944418'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:24.341421 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8a33c5ef-f86f-407f-8b4d-5fb9f1d797c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77b13775ee864057801505a005944418'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:24.362667 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:48:24.366969 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8a33c5ef-f86f-407f-8b4d-5fb9f1d797c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77b13775ee864057801505a005944418'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:24.385627 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1160/2307] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:48:24 2022] DELETE /identity/v3/users/77b13775ee864057801505a005944418 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:48:24.582147 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24439bb-adf9-4c16-880f-efc26db07635 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:24.582579 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24439bb-adf9-4c16-880f-efc26db07635 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:24.582765 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24439bb-adf9-4c16-880f-efc26db07635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:24.619058 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-e24439bb-adf9-4c16-880f-efc26db07635 None None] MFA Rules not processed for user `0c83c05bec4f456d8aa7d51732ddc8af`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:48:24.620949 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-e24439bb-adf9-4c16-880f-efc26db07635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:24.699235 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:48:24.718394 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1148/2308] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:48:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:48:24.873183 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a33c5ef-f86f-407f-8b4d-5fb9f1d797c1 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:24.875295 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-8a33c5ef-f86f-407f-8b4d-5fb9f1d797c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:25.045342 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a33c5ef-f86f-407f-8b4d-5fb9f1d797c1 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:25.051676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2a7ac-3860-431a-877e-72da7e7091fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:25.052566 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2a7ac-3860-431a-877e-72da7e7091fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:25.052865 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2a7ac-3860-431a-877e-72da7e7091fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:25.053161 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2a7ac-3860-431a-877e-72da7e7091fa admin admin] PATH_INFO: `/v3/projects/09128e2092e944b88023fc4a7c9d8411` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:25.054753 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c2a7ac-3860-431a-877e-72da7e7091fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=09128e2092e944b88023fc4a7c9d8411)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:25.056673 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c2a7ac-3860-431a-877e-72da7e7091fa admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:25.151546 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c3c2a7ac-3860-431a-877e-72da7e7091fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09128e2092e944b88023fc4a7c9d8411', 'request_id': 'req-c3c2a7ac-3860-431a-877e-72da7e7091fa'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:25.165860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c3c2a7ac-3860-431a-877e-72da7e7091fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09128e2092e944b88023fc4a7c9d8411', 'request_id': 'req-c3c2a7ac-3860-431a-877e-72da7e7091fa'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:25.179817 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c3c2a7ac-3860-431a-877e-72da7e7091fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09128e2092e944b88023fc4a7c9d8411', 'request_id': 'req-c3c2a7ac-3860-431a-877e-72da7e7091fa'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:25.208050 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c3c2a7ac-3860-431a-877e-72da7e7091fa admin admin] The token cache is being invalidate because project 09128e2092e944b88023fc4a7c9d8411 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:48:25.208361 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c3c2a7ac-3860-431a-877e-72da7e7091fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:25.209266 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c3c2a7ac-3860-431a-877e-72da7e7091fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:25.213608 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1161/2309] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:48:24 2022] DELETE /identity/v3/projects/09128e2092e944b88023fc4a7c9d8411 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:48:25.218465 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f418e72-9906-487d-b703-63c087038e9c None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:25.218984 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f418e72-9906-487d-b703-63c087038e9c None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:25.219201 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f418e72-9906-487d-b703-63c087038e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:25.289841 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-6f418e72-9906-487d-b703-63c087038e9c None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:48:25.291239 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-6f418e72-9906-487d-b703-63c087038e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:25.448823 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1149/2310] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:48:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:48:25.471041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73abb5c-5842-42fb-b51f-1b282f0636ce None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:25.471523 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73abb5c-5842-42fb-b51f-1b282f0636ce None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:25.472063 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73abb5c-5842-42fb-b51f-1b282f0636ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:25.496237 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-a73abb5c-5842-42fb-b51f-1b282f0636ce None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:48:25.498178 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-a73abb5c-5842-42fb-b51f-1b282f0636ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:25.526118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1162/2311] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:48:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:48:25.531998 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f418e72-9906-487d-b703-63c087038e9c None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:25.571677 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f418e72-9906-487d-b703-63c087038e9c None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:25.589226 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61677858-209e-4cad-a405-939079d5d17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:25.589489 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61677858-209e-4cad-a405-939079d5d17c admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:25.589719 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61677858-209e-4cad-a405-939079d5d17c admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:25.589895 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61677858-209e-4cad-a405-939079d5d17c admin admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:25.590652 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61677858-209e-4cad-a405-939079d5d17c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:48:25.590964 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61677858-209e-4cad-a405-939079d5d17c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:25.592642 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61677858-209e-4cad-a405-939079d5d17c admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:25.617908 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1150/2312] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:48:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:48:25.622597 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73abb5c-5842-42fb-b51f-1b282f0636ce None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:25.637559 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73abb5c-5842-42fb-b51f-1b282f0636ce None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:25.645974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b838852a-c306-447e-958e-98c72b12286c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:25.646884 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838852a-c306-447e-958e-98c72b12286c admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:25.648197 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838852a-c306-447e-958e-98c72b12286c admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:25.648197 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838852a-c306-447e-958e-98c72b12286c admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:25.648197 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b838852a-c306-447e-958e-98c72b12286c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:25.650412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b838852a-c306-447e-958e-98c72b12286c admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:25.691311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1163/2313] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:48:25 2022] POST /identity/v3/projects => generated 371 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:48:25.695374 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61677858-209e-4cad-a405-939079d5d17c admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:25.708326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61677858-209e-4cad-a405-939079d5d17c admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:25.884959 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff9193-2287-4601-ad49-2e0e49f19095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:25.885841 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff9193-2287-4601-ad49-2e0e49f19095 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:25.886172 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff9193-2287-4601-ad49-2e0e49f19095 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:25.888600 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff9193-2287-4601-ad49-2e0e49f19095 admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:25.888600 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff9193-2287-4601-ad49-2e0e49f19095 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:25.889619 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff9193-2287-4601-ad49-2e0e49f19095 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:25.919422 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1151/2314] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:48:25 2022] POST /identity/v3/users => generated 286 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:48:25.920236 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b838852a-c306-447e-958e-98c72b12286c admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:25.940207 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b838852a-c306-447e-958e-98c72b12286c admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:25.958488 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898468b-e385-4bcd-9d61-b43a4b4ad617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:25.959200 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898468b-e385-4bcd-9d61-b43a4b4ad617 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:25.959485 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898468b-e385-4bcd-9d61-b43a4b4ad617 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:25.959778 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898468b-e385-4bcd-9d61-b43a4b4ad617 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:25.960521 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898468b-e385-4bcd-9d61-b43a4b4ad617 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:25.963269 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898468b-e385-4bcd-9d61-b43a4b4ad617 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:25.990806 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff9193-2287-4601-ad49-2e0e49f19095 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:26.003165 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1164/2315] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:48:25 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:48:26.019778 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff9193-2287-4601-ad49-2e0e49f19095 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:26.036779 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4aab9a-95e9-4d6c-8429-d01edc65c98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:26.037519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4aab9a-95e9-4d6c-8429-d01edc65c98a admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:26.037806 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4aab9a-95e9-4d6c-8429-d01edc65c98a admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:26.038332 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4aab9a-95e9-4d6c-8429-d01edc65c98a admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:26.039247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4aab9a-95e9-4d6c-8429-d01edc65c98a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:26.041328 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4aab9a-95e9-4d6c-8429-d01edc65c98a admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:26.085304 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1152/2316] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:48:25 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:48:26.086628 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898468b-e385-4bcd-9d61-b43a4b4ad617 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:26.101970 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898468b-e385-4bcd-9d61-b43a4b4ad617 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:26.114545 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad3a652-8586-4555-8878-fbf8962329fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:26.116294 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3a652-8586-4555-8878-fbf8962329fe admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:26.116911 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3a652-8586-4555-8878-fbf8962329fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:26.117788 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3a652-8586-4555-8878-fbf8962329fe admin admin] PATH_INFO: `/v3/projects/fefd4cd82f63410198864e1afbf60ffc/users/a44316d784214d5da0607d0c43f11f72/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:26.161957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad3a652-8586-4555-8878-fbf8962329fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=fefd4cd82f63410198864e1afbf60ffc, user_id=a44316d784214d5da0607d0c43f11f72, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:26.166878 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad3a652-8586-4555-8878-fbf8962329fe admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:26.193191 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1165/2317] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:48:26 2022] PUT /identity/v3/projects/fefd4cd82f63410198864e1afbf60ffc/users/a44316d784214d5da0607d0c43f11f72/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:48:26.205471 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb0bc1-5c24-4f9a-ba25-b66e493a09df None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:26.205867 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb0bc1-5c24-4f9a-ba25-b66e493a09df None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:26.206259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb0bc1-5c24-4f9a-ba25-b66e493a09df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:26.250753 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-01eb0bc1-5c24-4f9a-ba25-b66e493a09df None None] MFA Rules not processed for user `a44316d784214d5da0607d0c43f11f72`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:48:26.252839 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-01eb0bc1-5c24-4f9a-ba25-b66e493a09df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:26.339914 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:48:26.374521 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1153/2318] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:48:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:48:26.383554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad3a652-8586-4555-8878-fbf8962329fe admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:26.385644 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-7ad3a652-8586-4555-8878-fbf8962329fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:26.439697 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:48:26.488038 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad3a652-8586-4555-8878-fbf8962329fe admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:26.494942 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce58c71-c398-4112-87fc-6911c5d206f0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4015c5ca2b8a438288820cd003f9a308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:26.497292 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce58c71-c398-4112-87fc-6911c5d206f0 service glance] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:26.497824 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce58c71-c398-4112-87fc-6911c5d206f0 service glance] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:26.498337 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce58c71-c398-4112-87fc-6911c5d206f0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:26.517874 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce58c71-c398-4112-87fc-6911c5d206f0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:26.519603 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce58c71-c398-4112-87fc-6911c5d206f0 service glance] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:26.545730 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1166/2319] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:48:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:48:28.072396 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b51bb8-dffa-4ad4-9414-4636dc7fb088 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:28.072868 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b51bb8-dffa-4ad4-9414-4636dc7fb088 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:28.072968 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b51bb8-dffa-4ad4-9414-4636dc7fb088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:28.091083 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-61b51bb8-dffa-4ad4-9414-4636dc7fb088 None None] MFA Rules not processed for user `c4535134181e489fb532f617264d5855`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:48:28.092225 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-61b51bb8-dffa-4ad4-9414-4636dc7fb088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:28.159231 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1154/2320] 10.222.0.20 () {58 vars in 967 bytes} [Wed Jul 6 13:48:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:48:28.179381 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce58c71-c398-4112-87fc-6911c5d206f0 service glance] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:28.182539 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-cce58c71-c398-4112-87fc-6911c5d206f0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:28.299833 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce58c71-c398-4112-87fc-6911c5d206f0 service glance] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:28.308531 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f8654-5961-4d34-bd45-0ddd9ac22121 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c1bea05c37646b0a05fa8dda0367ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:28.309570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f8654-5961-4d34-bd45-0ddd9ac22121 service swift] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:28.310011 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f8654-5961-4d34-bd45-0ddd9ac22121 service swift] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:28.310471 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f8654-5961-4d34-bd45-0ddd9ac22121 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:28.325167 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f8654-5961-4d34-bd45-0ddd9ac22121 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:28.327052 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f8654-5961-4d34-bd45-0ddd9ac22121 service swift] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:28.337947 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1167/2321] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jul 6 13:48:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 13:48:30.696360 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b51bb8-dffa-4ad4-9414-4636dc7fb088 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:30.697867 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-61b51bb8-dffa-4ad4-9414-4636dc7fb088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:30.774755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b51bb8-dffa-4ad4-9414-4636dc7fb088 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:30.779832 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee67cbc-9099-4aff-a2e9-684c1dbb8ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:30.780746 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee67cbc-9099-4aff-a2e9-684c1dbb8ae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:30.781057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee67cbc-9099-4aff-a2e9-684c1dbb8ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:30.781344 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee67cbc-9099-4aff-a2e9-684c1dbb8ae8 admin admin] PATH_INFO: `/v3/users/050d5eb22b2c45ee85eb8c118b9ef21e` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:30.782688 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee67cbc-9099-4aff-a2e9-684c1dbb8ae8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=050d5eb22b2c45ee85eb8c118b9ef21e)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:30.784651 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee67cbc-9099-4aff-a2e9-684c1dbb8ae8 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:30.852148 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-5ee67cbc-9099-4aff-a2e9-684c1dbb8ae8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '050d5eb22b2c45ee85eb8c118b9ef21e'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:30.859989 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-5ee67cbc-9099-4aff-a2e9-684c1dbb8ae8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '050d5eb22b2c45ee85eb8c118b9ef21e'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:30.861702 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-5ee67cbc-9099-4aff-a2e9-684c1dbb8ae8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '050d5eb22b2c45ee85eb8c118b9ef21e'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:30.868685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-5ee67cbc-9099-4aff-a2e9-684c1dbb8ae8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '050d5eb22b2c45ee85eb8c118b9ef21e'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:30.885510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-5ee67cbc-9099-4aff-a2e9-684c1dbb8ae8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '050d5eb22b2c45ee85eb8c118b9ef21e'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:30.891583 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1155/2322] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:48:30 2022] DELETE /identity/v3/users/050d5eb22b2c45ee85eb8c118b9ef21e => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:48:31.319778 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f8654-5961-4d34-bd45-0ddd9ac22121 service swift] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:31.331325 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f8654-5961-4d34-bd45-0ddd9ac22121 service swift] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:31.402162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43333d67-c139-4601-9771-92a386b28a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:31.403349 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43333d67-c139-4601-9771-92a386b28a30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:31.403658 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43333d67-c139-4601-9771-92a386b28a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:31.403939 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43333d67-c139-4601-9771-92a386b28a30 admin admin] PATH_INFO: `/v3/projects/2f9f86fa910c4054b31927ba26ea96b9` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:31.405170 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43333d67-c139-4601-9771-92a386b28a30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f9f86fa910c4054b31927ba26ea96b9)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:31.407070 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43333d67-c139-4601-9771-92a386b28a30 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:31.458893 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-43333d67-c139-4601-9771-92a386b28a30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f9f86fa910c4054b31927ba26ea96b9', 'request_id': 'req-43333d67-c139-4601-9771-92a386b28a30'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:31.463329 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:48:31.464628 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-43333d67-c139-4601-9771-92a386b28a30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f9f86fa910c4054b31927ba26ea96b9', 'request_id': 'req-43333d67-c139-4601-9771-92a386b28a30'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:31.473253 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-43333d67-c139-4601-9771-92a386b28a30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f9f86fa910c4054b31927ba26ea96b9', 'request_id': 'req-43333d67-c139-4601-9771-92a386b28a30'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:31.484264 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-43333d67-c139-4601-9771-92a386b28a30 admin admin] The token cache is being invalidate because project 2f9f86fa910c4054b31927ba26ea96b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:48:31.484539 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-43333d67-c139-4601-9771-92a386b28a30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:31.485210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-43333d67-c139-4601-9771-92a386b28a30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:48:31.487361 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1168/2323] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:48:31 2022] DELETE /identity/v3/projects/2f9f86fa910c4054b31927ba26ea96b9 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:48:31.493288 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1feb21-dab5-4b38-a025-1b7178f1824a None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:31.493577 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1feb21-dab5-4b38-a025-1b7178f1824a None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:31.493851 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1feb21-dab5-4b38-a025-1b7178f1824a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:31.512367 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-ff1feb21-dab5-4b38-a025-1b7178f1824a None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:48:31.513618 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-ff1feb21-dab5-4b38-a025-1b7178f1824a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:31.524724 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:48:31.577735 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1156/2324] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:48:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:48:31.586429 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e284d-21ef-4b04-ac3b-93502183d8e2 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:31.586719 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e284d-21ef-4b04-ac3b-93502183d8e2 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:31.587027 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e284d-21ef-4b04-ac3b-93502183d8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:31.606062 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-c67e284d-21ef-4b04-ac3b-93502183d8e2 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:48:31.607265 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-c67e284d-21ef-4b04-ac3b-93502183d8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:31.616466 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1169/2325] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:48:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:48:31.620863 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1feb21-dab5-4b38-a025-1b7178f1824a None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:31.631721 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1feb21-dab5-4b38-a025-1b7178f1824a None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:31.636259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51547189-7f40-4885-aff2-ea26bfd0ebf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:31.636913 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51547189-7f40-4885-aff2-ea26bfd0ebf3 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:31.637227 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51547189-7f40-4885-aff2-ea26bfd0ebf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:31.637502 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51547189-7f40-4885-aff2-ea26bfd0ebf3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:31.638226 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51547189-7f40-4885-aff2-ea26bfd0ebf3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:48:31.638531 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51547189-7f40-4885-aff2-ea26bfd0ebf3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:31.640145 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51547189-7f40-4885-aff2-ea26bfd0ebf3 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:31.652230 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1157/2326] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:48:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:48:31.656393 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e284d-21ef-4b04-ac3b-93502183d8e2 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:31.666102 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e284d-21ef-4b04-ac3b-93502183d8e2 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:31.670667 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c3b3c0-befc-4ffc-a2e6-19925ae21a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:31.671362 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3b3c0-befc-4ffc-a2e6-19925ae21a79 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:31.671641 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3b3c0-befc-4ffc-a2e6-19925ae21a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:31.672051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3b3c0-befc-4ffc-a2e6-19925ae21a79 admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:31.673530 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c3b3c0-befc-4ffc-a2e6-19925ae21a79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:31.676383 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c3b3c0-befc-4ffc-a2e6-19925ae21a79 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:31.702944 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1170/2327] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:48:31 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:48:31.707541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51547189-7f40-4885-aff2-ea26bfd0ebf3 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:31.718822 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51547189-7f40-4885-aff2-ea26bfd0ebf3 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:31.777119 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409fb92-9f84-495a-9004-833af82f32c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:31.777800 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409fb92-9f84-495a-9004-833af82f32c6 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:31.778082 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409fb92-9f84-495a-9004-833af82f32c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:31.778357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409fb92-9f84-495a-9004-833af82f32c6 admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:31.779093 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a409fb92-9f84-495a-9004-833af82f32c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:31.781207 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a409fb92-9f84-495a-9004-833af82f32c6 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:31.803751 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1158/2328] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:48:31 2022] POST /identity/v3/users => generated 284 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:48:31.807737 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c3b3c0-befc-4ffc-a2e6-19925ae21a79 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:31.819207 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c3b3c0-befc-4ffc-a2e6-19925ae21a79 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:31.823950 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e9905-89bc-4b81-937a-dbed206218df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:31.824603 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e9905-89bc-4b81-937a-dbed206218df admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:31.824882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e9905-89bc-4b81-937a-dbed206218df admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:31.825183 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e9905-89bc-4b81-937a-dbed206218df admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:31.825906 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e9905-89bc-4b81-937a-dbed206218df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:31.827854 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e9905-89bc-4b81-937a-dbed206218df admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:31.842671 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1171/2329] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:48:31.847324 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409fb92-9f84-495a-9004-833af82f32c6 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:31.857376 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409fb92-9f84-495a-9004-833af82f32c6 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:31.862781 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acedfc1f-70a4-4f09-9fd9-6a7a5b9ab089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:31.863440 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acedfc1f-70a4-4f09-9fd9-6a7a5b9ab089 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:31.863721 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acedfc1f-70a4-4f09-9fd9-6a7a5b9ab089 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:31.863996 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acedfc1f-70a4-4f09-9fd9-6a7a5b9ab089 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:31.864720 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acedfc1f-70a4-4f09-9fd9-6a7a5b9ab089 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:31.866343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acedfc1f-70a4-4f09-9fd9-6a7a5b9ab089 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:31.880656 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1159/2330] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:48:31.884881 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e9905-89bc-4b81-937a-dbed206218df admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:31.895447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e9905-89bc-4b81-937a-dbed206218df admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:31.900133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6016d4-d785-470b-8f6d-705d38045380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:31.901025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6016d4-d785-470b-8f6d-705d38045380 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:31.901330 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6016d4-d785-470b-8f6d-705d38045380 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:31.901609 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6016d4-d785-470b-8f6d-705d38045380 admin admin] PATH_INFO: `/v3/projects/e8f0f3bfff314f6dbb6e13e6778d12f7/users/4ea64c9fc1834ffcb13e1d1b6467ec4f/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:31.918167 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6016d4-d785-470b-8f6d-705d38045380 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8f0f3bfff314f6dbb6e13e6778d12f7, user_id=4ea64c9fc1834ffcb13e1d1b6467ec4f, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:31.921132 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6016d4-d785-470b-8f6d-705d38045380 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:31.945094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1172/2331] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:48:31 2022] PUT /identity/v3/projects/e8f0f3bfff314f6dbb6e13e6778d12f7/users/4ea64c9fc1834ffcb13e1d1b6467ec4f/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:48:31.947423 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528553b-3a5b-46af-8452-19f3051e096d None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:31.947735 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528553b-3a5b-46af-8452-19f3051e096d None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:31.948011 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528553b-3a5b-46af-8452-19f3051e096d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:31.979701 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-f528553b-3a5b-46af-8452-19f3051e096d None None] MFA Rules not processed for user `4ea64c9fc1834ffcb13e1d1b6467ec4f`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:48:31.980875 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-f528553b-3a5b-46af-8452-19f3051e096d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:32.058327 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1160/2332] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:48:31 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:48:32.070629 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6016d4-d785-470b-8f6d-705d38045380 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:32.072362 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-af6016d4-d785-470b-8f6d-705d38045380 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:32.156307 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6016d4-d785-470b-8f6d-705d38045380 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:32.160566 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40d4e9-5197-4dfb-8305-777e34a7a4ce service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4015c5ca2b8a438288820cd003f9a308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:32.161281 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40d4e9-5197-4dfb-8305-777e34a7a4ce service glance] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:32.161568 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40d4e9-5197-4dfb-8305-777e34a7a4ce service glance] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:32.161874 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40d4e9-5197-4dfb-8305-777e34a7a4ce service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:32.170418 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40d4e9-5197-4dfb-8305-777e34a7a4ce service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:32.171906 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40d4e9-5197-4dfb-8305-777e34a7a4ce service glance] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:32.182607 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1173/2333] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:48:32 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 13:48:37.955762 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528553b-3a5b-46af-8452-19f3051e096d None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:48:37.957246 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-f528553b-3a5b-46af-8452-19f3051e096d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:37.970367 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:48:38.015661 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528553b-3a5b-46af-8452-19f3051e096d None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:48:38.019963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f6597-682d-4af5-95ec-5a52add77d47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:48:38.020608 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f6597-682d-4af5-95ec-5a52add77d47 service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:48:38.020892 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f6597-682d-4af5-95ec-5a52add77d47 service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:48:38.021201 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f6597-682d-4af5-95ec-5a52add77d47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:48:38.022966 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-7b2f6597-682d-4af5-95ec-5a52add77d47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:48:38.104123 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f6597-682d-4af5-95ec-5a52add77d47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:48:38.105719 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f6597-682d-4af5-95ec-5a52add77d47 service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:48:38.129117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1161/2334] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:48:37 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:48:42.178301 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=67,keystone:INSERT=3,keystone:DELETE=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:48:48.113230 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=45 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:49:27.952981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40d4e9-5197-4dfb-8305-777e34a7a4ce service glance] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:27.955294 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-0a40d4e9-5197-4dfb-8305-777e34a7a4ce service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:28.065792 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40d4e9-5197-4dfb-8305-777e34a7a4ce service glance] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:28.071544 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f487136-0721-4f5c-b164-8e49a4f03a71 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '890d2ae150d647a18dae25159cf97484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:28.072130 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f487136-0721-4f5c-b164-8e49a4f03a71 service placement] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:28.072418 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f487136-0721-4f5c-b164-8e49a4f03a71 service placement] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:28.072749 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f487136-0721-4f5c-b164-8e49a4f03a71 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:28.074752 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-0f487136-0721-4f5c-b164-8e49a4f03a71 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:28.087500 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f487136-0721-4f5c-b164-8e49a4f03a71 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:28.089240 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f487136-0721-4f5c-b164-8e49a4f03a71 service placement] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:28.100202 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1174/2335] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:49:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:49:29.213737 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7a4fd6-8c25-479c-8abe-1303e0eaa876 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:29.214252 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7a4fd6-8c25-479c-8abe-1303e0eaa876 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:29.214357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7a4fd6-8c25-479c-8abe-1303e0eaa876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:29.231920 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-8f7a4fd6-8c25-479c-8abe-1303e0eaa876 None None] MFA Rules not processed for user `7b572a8f465048558fc969b85b2aec5b`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:49:29.233137 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-8f7a4fd6-8c25-479c-8abe-1303e0eaa876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:29.287171 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1162/2336] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:49:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:49:33.321730 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f487136-0721-4f5c-b164-8e49a4f03a71 service placement] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:33.323239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-0f487136-0721-4f5c-b164-8e49a4f03a71 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:33.396535 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f487136-0721-4f5c-b164-8e49a4f03a71 service placement] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:33.401125 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af78b9e-d7d8-47ea-9979-c07719f3c459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:33.401996 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af78b9e-d7d8-47ea-9979-c07719f3c459 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:33.402762 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af78b9e-d7d8-47ea-9979-c07719f3c459 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:33.402762 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af78b9e-d7d8-47ea-9979-c07719f3c459 admin admin] PATH_INFO: `/v3/users/a44316d784214d5da0607d0c43f11f72` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:33.403740 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af78b9e-d7d8-47ea-9979-c07719f3c459 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a44316d784214d5da0607d0c43f11f72)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:33.405699 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af78b9e-d7d8-47ea-9979-c07719f3c459 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:33.453896 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-9af78b9e-d7d8-47ea-9979-c07719f3c459 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a44316d784214d5da0607d0c43f11f72'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:33.461970 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-9af78b9e-d7d8-47ea-9979-c07719f3c459 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a44316d784214d5da0607d0c43f11f72'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:33.475068 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-9af78b9e-d7d8-47ea-9979-c07719f3c459 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a44316d784214d5da0607d0c43f11f72'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:33.475068 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-9af78b9e-d7d8-47ea-9979-c07719f3c459 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a44316d784214d5da0607d0c43f11f72'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:33.481974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:49:33.487889 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-9af78b9e-d7d8-47ea-9979-c07719f3c459 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a44316d784214d5da0607d0c43f11f72'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:33.494518 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1175/2337] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:49:33 2022] DELETE /identity/v3/users/a44316d784214d5da0607d0c43f11f72 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:49:33.498880 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175fd55-787d-46ca-b7ce-7af83599b0b5 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:33.499349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175fd55-787d-46ca-b7ce-7af83599b0b5 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:33.499466 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175fd55-787d-46ca-b7ce-7af83599b0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:33.516681 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-e175fd55-787d-46ca-b7ce-7af83599b0b5 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:49:33.517838 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-e175fd55-787d-46ca-b7ce-7af83599b0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:33.576160 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1163/2338] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:49:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:49:33.586054 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af78b9e-d7d8-47ea-9979-c07719f3c459 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:33.587419 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-9af78b9e-d7d8-47ea-9979-c07719f3c459 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:33.642330 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af78b9e-d7d8-47ea-9979-c07719f3c459 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:33.646873 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfedb33-1145-4af0-85b4-07f99eea90ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:33.647522 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfedb33-1145-4af0-85b4-07f99eea90ff service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:33.647792 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfedb33-1145-4af0-85b4-07f99eea90ff service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:33.648067 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfedb33-1145-4af0-85b4-07f99eea90ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:33.656848 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfedb33-1145-4af0-85b4-07f99eea90ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:33.658621 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfedb33-1145-4af0-85b4-07f99eea90ff service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:33.668652 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1176/2339] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:49:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:49:33.946914 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175fd55-787d-46ca-b7ce-7af83599b0b5 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:33.948765 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-e175fd55-787d-46ca-b7ce-7af83599b0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:33.964602 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175fd55-787d-46ca-b7ce-7af83599b0b5 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:33.970339 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27479d8e-cf4a-40d8-a5ee-3d79c01de1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:33.971206 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27479d8e-cf4a-40d8-a5ee-3d79c01de1cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:33.971485 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27479d8e-cf4a-40d8-a5ee-3d79c01de1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:33.971762 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27479d8e-cf4a-40d8-a5ee-3d79c01de1cd admin admin] PATH_INFO: `/v3/projects/fefd4cd82f63410198864e1afbf60ffc` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:33.973985 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27479d8e-cf4a-40d8-a5ee-3d79c01de1cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=fefd4cd82f63410198864e1afbf60ffc)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:33.975935 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27479d8e-cf4a-40d8-a5ee-3d79c01de1cd admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:34.022820 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-27479d8e-cf4a-40d8-a5ee-3d79c01de1cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fefd4cd82f63410198864e1afbf60ffc', 'request_id': 'req-27479d8e-cf4a-40d8-a5ee-3d79c01de1cd'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:34.028792 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-27479d8e-cf4a-40d8-a5ee-3d79c01de1cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fefd4cd82f63410198864e1afbf60ffc', 'request_id': 'req-27479d8e-cf4a-40d8-a5ee-3d79c01de1cd'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:34.037204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-27479d8e-cf4a-40d8-a5ee-3d79c01de1cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fefd4cd82f63410198864e1afbf60ffc', 'request_id': 'req-27479d8e-cf4a-40d8-a5ee-3d79c01de1cd'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:34.048479 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-27479d8e-cf4a-40d8-a5ee-3d79c01de1cd admin admin] The token cache is being invalidate because project fefd4cd82f63410198864e1afbf60ffc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:49:34.048860 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-27479d8e-cf4a-40d8-a5ee-3d79c01de1cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:34.049581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-27479d8e-cf4a-40d8-a5ee-3d79c01de1cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:34.052483 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1164/2340] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:49:33 2022] DELETE /identity/v3/projects/fefd4cd82f63410198864e1afbf60ffc => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:49:34.057379 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20efe6dc-1850-4fa2-b6d1-2b61c5b086bf None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:34.057668 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20efe6dc-1850-4fa2-b6d1-2b61c5b086bf None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:34.057945 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20efe6dc-1850-4fa2-b6d1-2b61c5b086bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:34.075048 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-20efe6dc-1850-4fa2-b6d1-2b61c5b086bf None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:49:34.076281 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-20efe6dc-1850-4fa2-b6d1-2b61c5b086bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:34.137577 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1177/2341] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:49:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:49:34.146027 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bc2bd-48f7-49cc-b8c0-dc246acd0863 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:34.146322 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bc2bd-48f7-49cc-b8c0-dc246acd0863 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:34.146610 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bc2bd-48f7-49cc-b8c0-dc246acd0863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:34.165763 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-fe9bc2bd-48f7-49cc-b8c0-dc246acd0863 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:49:34.166947 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-fe9bc2bd-48f7-49cc-b8c0-dc246acd0863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:34.174699 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1165/2342] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:49:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:49:34.179131 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20efe6dc-1850-4fa2-b6d1-2b61c5b086bf None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:34.188652 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20efe6dc-1850-4fa2-b6d1-2b61c5b086bf None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:34.193436 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cfa26a-7fd5-46fc-87d4-e655b655d5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:34.194086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfa26a-7fd5-46fc-87d4-e655b655d5b1 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:34.194395 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfa26a-7fd5-46fc-87d4-e655b655d5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:34.194629 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfa26a-7fd5-46fc-87d4-e655b655d5b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:34.195353 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cfa26a-7fd5-46fc-87d4-e655b655d5b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:49:34.195657 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cfa26a-7fd5-46fc-87d4-e655b655d5b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:34.197256 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cfa26a-7fd5-46fc-87d4-e655b655d5b1 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:34.213585 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1178/2343] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:49:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:49:34.214583 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9bc2bd-48f7-49cc-b8c0-dc246acd0863 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:34.224538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9bc2bd-48f7-49cc-b8c0-dc246acd0863 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:34.229639 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705d204-6b71-4185-b9a6-b7328979ae00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:34.230306 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705d204-6b71-4185-b9a6-b7328979ae00 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:34.230593 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705d204-6b71-4185-b9a6-b7328979ae00 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:34.230871 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705d204-6b71-4185-b9a6-b7328979ae00 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:34.231575 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5705d204-6b71-4185-b9a6-b7328979ae00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:34.233497 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5705d204-6b71-4185-b9a6-b7328979ae00 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:34.254269 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:49:34.262763 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1166/2344] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:49:34 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:49:34.266447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cfa26a-7fd5-46fc-87d4-e655b655d5b1 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:34.276743 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cfa26a-7fd5-46fc-87d4-e655b655d5b1 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:34.335787 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9eeb5-8df3-4930-b0a7-b2296a261659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:34.336470 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9eeb5-8df3-4930-b0a7-b2296a261659 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:34.336845 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9eeb5-8df3-4930-b0a7-b2296a261659 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:34.337154 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9eeb5-8df3-4930-b0a7-b2296a261659 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:34.337857 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e9eeb5-8df3-4930-b0a7-b2296a261659 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:34.339740 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e9eeb5-8df3-4930-b0a7-b2296a261659 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:34.363185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1179/2345] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:49:34 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:49:34.364320 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705d204-6b71-4185-b9a6-b7328979ae00 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:34.374792 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705d204-6b71-4185-b9a6-b7328979ae00 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:34.380167 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f18e9-3ad8-4489-ba3d-abbf4cc7e530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:34.380814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f18e9-3ad8-4489-ba3d-abbf4cc7e530 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:34.381203 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f18e9-3ad8-4489-ba3d-abbf4cc7e530 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:34.381386 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f18e9-3ad8-4489-ba3d-abbf4cc7e530 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:34.382103 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f18e9-3ad8-4489-ba3d-abbf4cc7e530 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:34.383695 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f18e9-3ad8-4489-ba3d-abbf4cc7e530 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:34.394232 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1167/2346] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:49:34.398662 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9eeb5-8df3-4930-b0a7-b2296a261659 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:34.408351 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9eeb5-8df3-4930-b0a7-b2296a261659 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:34.413582 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e1c91-8fd0-4f18-98f7-a1804d186cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:34.414239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e1c91-8fd0-4f18-98f7-a1804d186cfb admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:34.414522 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e1c91-8fd0-4f18-98f7-a1804d186cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:34.414851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e1c91-8fd0-4f18-98f7-a1804d186cfb admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:34.415630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e1c91-8fd0-4f18-98f7-a1804d186cfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:34.417424 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e1c91-8fd0-4f18-98f7-a1804d186cfb admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:34.419978 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:49:34.435146 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1180/2347] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:49:34.436345 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f18e9-3ad8-4489-ba3d-abbf4cc7e530 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:34.447533 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f18e9-3ad8-4489-ba3d-abbf4cc7e530 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:34.452701 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab864fe-a887-4a2f-bedf-27318563552b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:34.453659 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab864fe-a887-4a2f-bedf-27318563552b admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:34.453947 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab864fe-a887-4a2f-bedf-27318563552b admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:34.454227 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab864fe-a887-4a2f-bedf-27318563552b admin admin] PATH_INFO: `/v3/projects/ef672f4b33994d7995f60a9d3bddfe7e/users/c7a4da8badf349418980ca5760c18455/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:34.469731 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab864fe-a887-4a2f-bedf-27318563552b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef672f4b33994d7995f60a9d3bddfe7e, user_id=c7a4da8badf349418980ca5760c18455, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:34.472574 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab864fe-a887-4a2f-bedf-27318563552b admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:34.511633 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1168/2348] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:49:34 2022] PUT /identity/v3/projects/ef672f4b33994d7995f60a9d3bddfe7e/users/c7a4da8badf349418980ca5760c18455/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:49:34.514642 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48333bb-a46e-4269-acde-004a984df6c0 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:34.515008 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48333bb-a46e-4269-acde-004a984df6c0 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:34.515364 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48333bb-a46e-4269-acde-004a984df6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:34.563366 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-f48333bb-a46e-4269-acde-004a984df6c0 None None] MFA Rules not processed for user `c7a4da8badf349418980ca5760c18455`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:49:34.564567 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-f48333bb-a46e-4269-acde-004a984df6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:34.637329 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1181/2349] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:49:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:49:34.650716 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab864fe-a887-4a2f-bedf-27318563552b admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:34.652366 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-1ab864fe-a887-4a2f-bedf-27318563552b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:34.711205 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab864fe-a887-4a2f-bedf-27318563552b admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:34.714850 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b131c57-b394-417b-aa33-1a9c7e0f09ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:34.715494 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b131c57-b394-417b-aa33-1a9c7e0f09ed service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:34.715777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b131c57-b394-417b-aa33-1a9c7e0f09ed service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:34.716052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b131c57-b394-417b-aa33-1a9c7e0f09ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:34.724723 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b131c57-b394-417b-aa33-1a9c7e0f09ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:34.726241 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b131c57-b394-417b-aa33-1a9c7e0f09ed service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:34.734893 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1169/2350] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:49:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:49:34.759389 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48333bb-a46e-4269-acde-004a984df6c0 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:34.823254 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48333bb-a46e-4269-acde-004a984df6c0 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:34.827770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa3bc8-068c-4cb3-993c-a528975093df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:34.828633 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa3bc8-068c-4cb3-993c-a528975093df admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:34.828979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa3bc8-068c-4cb3-993c-a528975093df admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:34.829269 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa3bc8-068c-4cb3-993c-a528975093df admin admin] PATH_INFO: `/v3/users/c7a4da8badf349418980ca5760c18455` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:34.830480 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aa3bc8-068c-4cb3-993c-a528975093df admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7a4da8badf349418980ca5760c18455)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:34.833145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aa3bc8-068c-4cb3-993c-a528975093df admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:34.878658 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-d7aa3bc8-068c-4cb3-993c-a528975093df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7a4da8badf349418980ca5760c18455'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:34.886219 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-d7aa3bc8-068c-4cb3-993c-a528975093df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7a4da8badf349418980ca5760c18455'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:34.892944 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-d7aa3bc8-068c-4cb3-993c-a528975093df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7a4da8badf349418980ca5760c18455'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:34.893599 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-d7aa3bc8-068c-4cb3-993c-a528975093df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7a4da8badf349418980ca5760c18455'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:34.905405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-d7aa3bc8-068c-4cb3-993c-a528975093df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7a4da8badf349418980ca5760c18455'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:34.911201 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1182/2351] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:49:34 2022] DELETE /identity/v3/users/c7a4da8badf349418980ca5760c18455 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:49:34.916083 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b118283-226c-4abc-b41a-176ebb8a5790 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:34.916375 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b118283-226c-4abc-b41a-176ebb8a5790 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:34.916654 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b118283-226c-4abc-b41a-176ebb8a5790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:34.935209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-8b118283-226c-4abc-b41a-176ebb8a5790 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:49:34.936398 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-8b118283-226c-4abc-b41a-176ebb8a5790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:34.995653 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1170/2352] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:49:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:49:35.006226 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa3bc8-068c-4cb3-993c-a528975093df admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:35.007649 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-d7aa3bc8-068c-4cb3-993c-a528975093df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:35.017418 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:49:35.068467 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa3bc8-068c-4cb3-993c-a528975093df admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:35.072074 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3340f9c-9137-459b-aae3-2dff043e190c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:35.072718 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3340f9c-9137-459b-aae3-2dff043e190c service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:35.073020 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3340f9c-9137-459b-aae3-2dff043e190c service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:35.073305 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3340f9c-9137-459b-aae3-2dff043e190c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:35.082760 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3340f9c-9137-459b-aae3-2dff043e190c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:35.084184 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3340f9c-9137-459b-aae3-2dff043e190c service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:35.093563 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1183/2353] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:49:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:49:35.360424 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b118283-226c-4abc-b41a-176ebb8a5790 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:35.362461 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-8b118283-226c-4abc-b41a-176ebb8a5790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:35.375789 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b118283-226c-4abc-b41a-176ebb8a5790 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:35.380928 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c3929-13c9-468c-b684-d69896fc29f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:35.381435 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c3929-13c9-468c-b684-d69896fc29f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:35.381625 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c3929-13c9-468c-b684-d69896fc29f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:35.381965 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c3929-13c9-468c-b684-d69896fc29f9 admin admin] PATH_INFO: `/v3/projects/ef672f4b33994d7995f60a9d3bddfe7e` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:35.383116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c3929-13c9-468c-b684-d69896fc29f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef672f4b33994d7995f60a9d3bddfe7e)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:35.385197 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c3929-13c9-468c-b684-d69896fc29f9 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:35.432939 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a22c3929-13c9-468c-b684-d69896fc29f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef672f4b33994d7995f60a9d3bddfe7e', 'request_id': 'req-a22c3929-13c9-468c-b684-d69896fc29f9'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:35.438932 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a22c3929-13c9-468c-b684-d69896fc29f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef672f4b33994d7995f60a9d3bddfe7e', 'request_id': 'req-a22c3929-13c9-468c-b684-d69896fc29f9'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:35.445763 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a22c3929-13c9-468c-b684-d69896fc29f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef672f4b33994d7995f60a9d3bddfe7e', 'request_id': 'req-a22c3929-13c9-468c-b684-d69896fc29f9'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:35.447838 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:49:35.458143 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a22c3929-13c9-468c-b684-d69896fc29f9 admin admin] The token cache is being invalidate because project ef672f4b33994d7995f60a9d3bddfe7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:49:35.458422 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a22c3929-13c9-468c-b684-d69896fc29f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:35.459037 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a22c3929-13c9-468c-b684-d69896fc29f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:35.460563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1171/2354] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:49:35 2022] DELETE /identity/v3/projects/ef672f4b33994d7995f60a9d3bddfe7e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:49:35.465485 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218222d-ac26-4863-be2e-832d55fc5ec1 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:35.465964 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218222d-ac26-4863-be2e-832d55fc5ec1 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:35.466083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218222d-ac26-4863-be2e-832d55fc5ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:35.483383 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-e218222d-ac26-4863-be2e-832d55fc5ec1 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:49:35.484553 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-e218222d-ac26-4863-be2e-832d55fc5ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:35.544731 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1184/2355] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:49:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:49:35.553606 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad7a13-3e1d-4989-ba6c-7b8fd637b8b3 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:35.554357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad7a13-3e1d-4989-ba6c-7b8fd637b8b3 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:35.554357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad7a13-3e1d-4989-ba6c-7b8fd637b8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:35.574127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-aaad7a13-3e1d-4989-ba6c-7b8fd637b8b3 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:49:35.575351 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-aaad7a13-3e1d-4989-ba6c-7b8fd637b8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:35.583285 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1172/2356] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:49:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:49:35.587520 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218222d-ac26-4863-be2e-832d55fc5ec1 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:35.596914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218222d-ac26-4863-be2e-832d55fc5ec1 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:35.602439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23e7d3-cc5b-4954-8a80-87b0ed8e3b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:35.603323 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23e7d3-cc5b-4954-8a80-87b0ed8e3b90 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:35.603560 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23e7d3-cc5b-4954-8a80-87b0ed8e3b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:35.603842 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23e7d3-cc5b-4954-8a80-87b0ed8e3b90 admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:35.604727 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b23e7d3-cc5b-4954-8a80-87b0ed8e3b90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:49:35.605076 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b23e7d3-cc5b-4954-8a80-87b0ed8e3b90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:35.606808 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b23e7d3-cc5b-4954-8a80-87b0ed8e3b90 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:35.619670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1185/2357] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:49:35.625410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad7a13-3e1d-4989-ba6c-7b8fd637b8b3 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:35.635111 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad7a13-3e1d-4989-ba6c-7b8fd637b8b3 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:35.639806 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b3095-5d12-404f-b572-8f9e8530fe34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:35.640458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b3095-5d12-404f-b572-8f9e8530fe34 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:35.640743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b3095-5d12-404f-b572-8f9e8530fe34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:35.641043 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b3095-5d12-404f-b572-8f9e8530fe34 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:35.641743 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3b3095-5d12-404f-b572-8f9e8530fe34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:35.643673 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3b3095-5d12-404f-b572-8f9e8530fe34 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:35.670109 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1173/2358] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:49:35 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:49:35.674143 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23e7d3-cc5b-4954-8a80-87b0ed8e3b90 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:35.685584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23e7d3-cc5b-4954-8a80-87b0ed8e3b90 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:35.744548 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fed3c-9ff7-432a-8155-112c76710d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:35.745406 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fed3c-9ff7-432a-8155-112c76710d45 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:35.745692 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fed3c-9ff7-432a-8155-112c76710d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:35.745963 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fed3c-9ff7-432a-8155-112c76710d45 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:35.746847 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9fed3c-9ff7-432a-8155-112c76710d45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:35.748753 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9fed3c-9ff7-432a-8155-112c76710d45 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:35.769438 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1186/2359] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:49:35 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:49:35.774164 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b3095-5d12-404f-b572-8f9e8530fe34 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:35.784864 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b3095-5d12-404f-b572-8f9e8530fe34 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:35.795142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1db25a-7720-487a-a5a0-a7324cd3e6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:35.795836 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1db25a-7720-487a-a5a0-a7324cd3e6f3 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:35.796127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1db25a-7720-487a-a5a0-a7324cd3e6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:35.796410 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1db25a-7720-487a-a5a0-a7324cd3e6f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:35.797184 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1db25a-7720-487a-a5a0-a7324cd3e6f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:35.798858 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1db25a-7720-487a-a5a0-a7324cd3e6f3 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:35.811165 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1174/2360] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:49:35.815821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fed3c-9ff7-432a-8155-112c76710d45 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:35.827366 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fed3c-9ff7-432a-8155-112c76710d45 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:35.835535 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2668a92-61db-4ef6-8fc7-b37c911f38e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:35.836326 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2668a92-61db-4ef6-8fc7-b37c911f38e1 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:35.836737 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2668a92-61db-4ef6-8fc7-b37c911f38e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:35.837285 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2668a92-61db-4ef6-8fc7-b37c911f38e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:35.838160 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2668a92-61db-4ef6-8fc7-b37c911f38e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:35.839988 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2668a92-61db-4ef6-8fc7-b37c911f38e1 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:35.856769 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1187/2361] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:49:35.857665 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1db25a-7720-487a-a5a0-a7324cd3e6f3 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:35.870168 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1db25a-7720-487a-a5a0-a7324cd3e6f3 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:35.875008 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e77817-2288-4aee-bdf5-80b8056274ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:35.875902 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e77817-2288-4aee-bdf5-80b8056274ca admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:35.876191 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e77817-2288-4aee-bdf5-80b8056274ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:35.876470 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e77817-2288-4aee-bdf5-80b8056274ca admin admin] PATH_INFO: `/v3/projects/ab7b8900cde9447dbff76a7cd1a36673/users/f1931d1b6dee4db9b35bc95c348f7eaf/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:35.890960 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e77817-2288-4aee-bdf5-80b8056274ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab7b8900cde9447dbff76a7cd1a36673, user_id=f1931d1b6dee4db9b35bc95c348f7eaf, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:35.893652 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e77817-2288-4aee-bdf5-80b8056274ca admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:35.908577 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1175/2362] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:49:35 2022] PUT /identity/v3/projects/ab7b8900cde9447dbff76a7cd1a36673/users/f1931d1b6dee4db9b35bc95c348f7eaf/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:49:35.919155 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f828fb-d6eb-4daa-a857-b462f66e6501 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:35.919448 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f828fb-d6eb-4daa-a857-b462f66e6501 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:35.919729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f828fb-d6eb-4daa-a857-b462f66e6501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:35.946420 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-98f828fb-d6eb-4daa-a857-b462f66e6501 None None] MFA Rules not processed for user `f1931d1b6dee4db9b35bc95c348f7eaf`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:49:35.947599 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-98f828fb-d6eb-4daa-a857-b462f66e6501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:35.951520 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:49:36.013159 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1188/2363] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:49:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:49:36.024777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e77817-2288-4aee-bdf5-80b8056274ca admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:36.026455 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-05e77817-2288-4aee-bdf5-80b8056274ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:36.087261 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e77817-2288-4aee-bdf5-80b8056274ca admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:36.090884 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7122959-e863-4a6b-bf5b-850cd948c430 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:36.091526 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7122959-e863-4a6b-bf5b-850cd948c430 service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:36.091809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7122959-e863-4a6b-bf5b-850cd948c430 service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:36.092089 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7122959-e863-4a6b-bf5b-850cd948c430 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:36.101172 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7122959-e863-4a6b-bf5b-850cd948c430 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:36.102603 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7122959-e863-4a6b-bf5b-850cd948c430 service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:36.111538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1176/2364] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:49:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:49:46.010260 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=25 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:49:46.107418 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=56,keystone:DELETE=1,keystone:INSERT=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:49:46.488525 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f828fb-d6eb-4daa-a857-b462f66e6501 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:46.490032 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-98f828fb-d6eb-4daa-a857-b462f66e6501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:46.562959 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f828fb-d6eb-4daa-a857-b462f66e6501 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:46.568194 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931c7bc-c27b-423b-a6fb-45616cce9949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:46.569218 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931c7bc-c27b-423b-a6fb-45616cce9949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:46.569597 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931c7bc-c27b-423b-a6fb-45616cce9949 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:46.569974 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931c7bc-c27b-423b-a6fb-45616cce9949 admin admin] PATH_INFO: `/v3/users/4ea64c9fc1834ffcb13e1d1b6467ec4f` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:46.571256 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2931c7bc-c27b-423b-a6fb-45616cce9949 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ea64c9fc1834ffcb13e1d1b6467ec4f)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:46.573309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2931c7bc-c27b-423b-a6fb-45616cce9949 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:46.637926 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-2931c7bc-c27b-423b-a6fb-45616cce9949 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ea64c9fc1834ffcb13e1d1b6467ec4f'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:46.647994 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-2931c7bc-c27b-423b-a6fb-45616cce9949 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ea64c9fc1834ffcb13e1d1b6467ec4f'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:46.656935 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-2931c7bc-c27b-423b-a6fb-45616cce9949 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ea64c9fc1834ffcb13e1d1b6467ec4f'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:46.659066 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-2931c7bc-c27b-423b-a6fb-45616cce9949 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ea64c9fc1834ffcb13e1d1b6467ec4f'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:46.675025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-2931c7bc-c27b-423b-a6fb-45616cce9949 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ea64c9fc1834ffcb13e1d1b6467ec4f'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:46.683478 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1189/2365] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:49:46 2022] DELETE /identity/v3/users/4ea64c9fc1834ffcb13e1d1b6467ec4f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:49:46.688351 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca013b15-78ff-4e23-852d-473e4a27a240 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:46.688746 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca013b15-78ff-4e23-852d-473e4a27a240 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:46.689402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca013b15-78ff-4e23-852d-473e4a27a240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:46.710503 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-ca013b15-78ff-4e23-852d-473e4a27a240 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:49:46.711703 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-ca013b15-78ff-4e23-852d-473e4a27a240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:46.789038 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1177/2366] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:49:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:49:46.797515 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931c7bc-c27b-423b-a6fb-45616cce9949 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:46.799250 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-2931c7bc-c27b-423b-a6fb-45616cce9949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:46.870047 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931c7bc-c27b-423b-a6fb-45616cce9949 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:46.874145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e8a77-f9bc-4b78-9821-10c3f499efdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:46.874870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e8a77-f9bc-4b78-9821-10c3f499efdd service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:46.875230 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e8a77-f9bc-4b78-9821-10c3f499efdd service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:46.875853 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e8a77-f9bc-4b78-9821-10c3f499efdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:46.885878 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e8a77-f9bc-4b78-9821-10c3f499efdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:46.887279 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e8a77-f9bc-4b78-9821-10c3f499efdd service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:46.901304 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1190/2367] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:49:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:49:47.242384 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca013b15-78ff-4e23-852d-473e4a27a240 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:47.245039 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-ca013b15-78ff-4e23-852d-473e4a27a240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:47.267129 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca013b15-78ff-4e23-852d-473e4a27a240 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:47.273741 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a1bc84-5d8a-45f5-8c77-ea0792cb159a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:47.274694 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1bc84-5d8a-45f5-8c77-ea0792cb159a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:47.274904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1bc84-5d8a-45f5-8c77-ea0792cb159a admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:47.275239 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1bc84-5d8a-45f5-8c77-ea0792cb159a admin admin] PATH_INFO: `/v3/projects/e8f0f3bfff314f6dbb6e13e6778d12f7` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:47.276552 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a1bc84-5d8a-45f5-8c77-ea0792cb159a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8f0f3bfff314f6dbb6e13e6778d12f7)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:47.279119 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a1bc84-5d8a-45f5-8c77-ea0792cb159a admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:47.334854 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-c8a1bc84-5d8a-45f5-8c77-ea0792cb159a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8f0f3bfff314f6dbb6e13e6778d12f7', 'request_id': 'req-c8a1bc84-5d8a-45f5-8c77-ea0792cb159a'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:47.341423 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-c8a1bc84-5d8a-45f5-8c77-ea0792cb159a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8f0f3bfff314f6dbb6e13e6778d12f7', 'request_id': 'req-c8a1bc84-5d8a-45f5-8c77-ea0792cb159a'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:47.349155 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-c8a1bc84-5d8a-45f5-8c77-ea0792cb159a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8f0f3bfff314f6dbb6e13e6778d12f7', 'request_id': 'req-c8a1bc84-5d8a-45f5-8c77-ea0792cb159a'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:47.360171 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-c8a1bc84-5d8a-45f5-8c77-ea0792cb159a admin admin] The token cache is being invalidate because project e8f0f3bfff314f6dbb6e13e6778d12f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:49:47.360460 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-c8a1bc84-5d8a-45f5-8c77-ea0792cb159a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:47.361120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-c8a1bc84-5d8a-45f5-8c77-ea0792cb159a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:47.362665 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1178/2368] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:49:47 2022] DELETE /identity/v3/projects/e8f0f3bfff314f6dbb6e13e6778d12f7 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:49:47.368081 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ba7d6-aeff-441f-a3d5-89bd3b8826d1 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:47.368570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ba7d6-aeff-441f-a3d5-89bd3b8826d1 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:47.368791 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ba7d6-aeff-441f-a3d5-89bd3b8826d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:47.386993 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-366ba7d6-aeff-441f-a3d5-89bd3b8826d1 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:49:47.388201 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-366ba7d6-aeff-441f-a3d5-89bd3b8826d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:47.407027 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:49:47.453514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1191/2369] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:49:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:49:47.462186 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22472e5c-2bdd-4e20-a40f-fa615784b300 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:47.462581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22472e5c-2bdd-4e20-a40f-fa615784b300 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:47.462867 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22472e5c-2bdd-4e20-a40f-fa615784b300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:47.481932 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-22472e5c-2bdd-4e20-a40f-fa615784b300 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:49:47.483160 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-22472e5c-2bdd-4e20-a40f-fa615784b300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:47.491917 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1179/2370] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:49:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:49:47.496562 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ba7d6-aeff-441f-a3d5-89bd3b8826d1 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:47.508870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ba7d6-aeff-441f-a3d5-89bd3b8826d1 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:47.513792 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf235e2-5382-4cfd-990e-2cad89f60565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:47.514468 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf235e2-5382-4cfd-990e-2cad89f60565 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:47.514752 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf235e2-5382-4cfd-990e-2cad89f60565 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:47.515026 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf235e2-5382-4cfd-990e-2cad89f60565 admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:47.515767 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf235e2-5382-4cfd-990e-2cad89f60565 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:49:47.516072 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf235e2-5382-4cfd-990e-2cad89f60565 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:47.517754 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf235e2-5382-4cfd-990e-2cad89f60565 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:47.535837 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1192/2371] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:49:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:49:47.537523 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22472e5c-2bdd-4e20-a40f-fa615784b300 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:47.547816 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22472e5c-2bdd-4e20-a40f-fa615784b300 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:47.553595 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7551c-3569-4057-a9b2-d1aa7273bf97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:47.554303 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7551c-3569-4057-a9b2-d1aa7273bf97 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:47.554586 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7551c-3569-4057-a9b2-d1aa7273bf97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:47.554863 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7551c-3569-4057-a9b2-d1aa7273bf97 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:47.555646 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b7551c-3569-4057-a9b2-d1aa7273bf97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:47.557738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b7551c-3569-4057-a9b2-d1aa7273bf97 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:47.591398 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1180/2372] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:49:47 2022] POST /identity/v3/projects => generated 391 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:49:47.592411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf235e2-5382-4cfd-990e-2cad89f60565 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:47.603724 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf235e2-5382-4cfd-990e-2cad89f60565 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:47.666119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c8db0-9fb2-4c9f-bb82-0157d0dc8187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:47.666803 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c8db0-9fb2-4c9f-bb82-0157d0dc8187 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:47.667108 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c8db0-9fb2-4c9f-bb82-0157d0dc8187 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:47.667384 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c8db0-9fb2-4c9f-bb82-0157d0dc8187 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:47.668095 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244c8db0-9fb2-4c9f-bb82-0157d0dc8187 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:47.670061 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244c8db0-9fb2-4c9f-bb82-0157d0dc8187 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:47.691863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1193/2373] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:49:47 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:49:47.696052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7551c-3569-4057-a9b2-d1aa7273bf97 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:47.706575 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7551c-3569-4057-a9b2-d1aa7273bf97 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:47.711518 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425dc0f8-7ce9-4110-a974-7406ba49bfb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:47.712193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425dc0f8-7ce9-4110-a974-7406ba49bfb3 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:47.712477 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425dc0f8-7ce9-4110-a974-7406ba49bfb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:47.712765 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425dc0f8-7ce9-4110-a974-7406ba49bfb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:47.713550 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425dc0f8-7ce9-4110-a974-7406ba49bfb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:47.715152 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425dc0f8-7ce9-4110-a974-7406ba49bfb3 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:47.727110 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1181/2374] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:49:47.730571 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c8db0-9fb2-4c9f-bb82-0157d0dc8187 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:47.742051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c8db0-9fb2-4c9f-bb82-0157d0dc8187 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:47.746903 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2f94c-ffac-4b07-b5b0-0354b0d40b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:47.747571 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2f94c-ffac-4b07-b5b0-0354b0d40b7d admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:47.747857 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2f94c-ffac-4b07-b5b0-0354b0d40b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:47.748140 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2f94c-ffac-4b07-b5b0-0354b0d40b7d admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:47.748867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a2f94c-ffac-4b07-b5b0-0354b0d40b7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:47.750502 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a2f94c-ffac-4b07-b5b0-0354b0d40b7d admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:47.761789 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1194/2375] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:49:47.765877 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425dc0f8-7ce9-4110-a974-7406ba49bfb3 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:47.776744 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425dc0f8-7ce9-4110-a974-7406ba49bfb3 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:47.781797 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fc5a2-fb89-4586-a8f1-9ce8ecee5cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:47.782715 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fc5a2-fb89-4586-a8f1-9ce8ecee5cec admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:47.783027 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fc5a2-fb89-4586-a8f1-9ce8ecee5cec admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:47.783305 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fc5a2-fb89-4586-a8f1-9ce8ecee5cec admin admin] PATH_INFO: `/v3/projects/0f986cf8068f483c84f4061d69dbf1fa/users/8d6f31a080154a8dbdfda23b1c890767/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:47.798280 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5fc5a2-fb89-4586-a8f1-9ce8ecee5cec admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f986cf8068f483c84f4061d69dbf1fa, user_id=8d6f31a080154a8dbdfda23b1c890767, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:47.802362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5fc5a2-fb89-4586-a8f1-9ce8ecee5cec admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:47.816701 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1182/2376] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:49:47 2022] PUT /identity/v3/projects/0f986cf8068f483c84f4061d69dbf1fa/users/8d6f31a080154a8dbdfda23b1c890767/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:49:47.828405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee1158-7795-46f0-8faa-8155098d2236 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:47.828696 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee1158-7795-46f0-8faa-8155098d2236 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:47.828978 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee1158-7795-46f0-8faa-8155098d2236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:47.859446 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-d1ee1158-7795-46f0-8faa-8155098d2236 None None] MFA Rules not processed for user `8d6f31a080154a8dbdfda23b1c890767`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:49:47.861280 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-d1ee1158-7795-46f0-8faa-8155098d2236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:47.939622 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1195/2377] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:49:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:49:47.951860 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fc5a2-fb89-4586-a8f1-9ce8ecee5cec admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:47.954219 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-df5fc5a2-fb89-4586-a8f1-9ce8ecee5cec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:47.981879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:49:48.022611 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fc5a2-fb89-4586-a8f1-9ce8ecee5cec admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:48.027436 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3d61d-ac4e-4189-ba18-00b531bc4e67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:48.028289 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3d61d-ac4e-4189-ba18-00b531bc4e67 service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:48.028726 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3d61d-ac4e-4189-ba18-00b531bc4e67 service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:48.029182 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3d61d-ac4e-4189-ba18-00b531bc4e67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:48.039328 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f3d61d-ac4e-4189-ba18-00b531bc4e67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:48.041057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f3d61d-ac4e-4189-ba18-00b531bc4e67 service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:48.053556 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1183/2378] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:49:47 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:49:54.909923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ee1158-7795-46f0-8faa-8155098d2236 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:54.911861 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-d1ee1158-7795-46f0-8faa-8155098d2236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:54.938161 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:49:54.988642 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ee1158-7795-46f0-8faa-8155098d2236 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:54.993087 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb72fe8-870f-4ec1-83b0-98678cc98955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:54.993953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb72fe8-870f-4ec1-83b0-98678cc98955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:54.994232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb72fe8-870f-4ec1-83b0-98678cc98955 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:54.994506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb72fe8-870f-4ec1-83b0-98678cc98955 admin admin] PATH_INFO: `/v3/users/f1931d1b6dee4db9b35bc95c348f7eaf` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:54.995683 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb72fe8-870f-4ec1-83b0-98678cc98955 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1931d1b6dee4db9b35bc95c348f7eaf)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:54.997623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb72fe8-870f-4ec1-83b0-98678cc98955 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:55.044164 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-fdb72fe8-870f-4ec1-83b0-98678cc98955 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1931d1b6dee4db9b35bc95c348f7eaf'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:55.052249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-fdb72fe8-870f-4ec1-83b0-98678cc98955 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1931d1b6dee4db9b35bc95c348f7eaf'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:55.059091 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-fdb72fe8-870f-4ec1-83b0-98678cc98955 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1931d1b6dee4db9b35bc95c348f7eaf'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:55.059733 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-fdb72fe8-870f-4ec1-83b0-98678cc98955 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1931d1b6dee4db9b35bc95c348f7eaf'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:55.072888 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-fdb72fe8-870f-4ec1-83b0-98678cc98955 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1931d1b6dee4db9b35bc95c348f7eaf'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:55.078801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1196/2379] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:49:54 2022] DELETE /identity/v3/users/f1931d1b6dee4db9b35bc95c348f7eaf => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:49:55.084078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08ba17-1bf5-4520-9302-96d908ac03de None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:55.084529 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08ba17-1bf5-4520-9302-96d908ac03de None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:55.084623 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08ba17-1bf5-4520-9302-96d908ac03de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:55.102136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-8a08ba17-1bf5-4520-9302-96d908ac03de None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:49:55.103322 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-8a08ba17-1bf5-4520-9302-96d908ac03de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:55.160671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1184/2380] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:49:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:49:55.170143 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb72fe8-870f-4ec1-83b0-98678cc98955 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:55.171803 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-fdb72fe8-870f-4ec1-83b0-98678cc98955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:55.229334 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb72fe8-870f-4ec1-83b0-98678cc98955 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:55.232980 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0404a-51a7-4e3e-a551-bff8b1747da3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:55.233648 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0404a-51a7-4e3e-a551-bff8b1747da3 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:55.233935 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0404a-51a7-4e3e-a551-bff8b1747da3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:55.234215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0404a-51a7-4e3e-a551-bff8b1747da3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:55.243670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0404a-51a7-4e3e-a551-bff8b1747da3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:55.245128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0404a-51a7-4e3e-a551-bff8b1747da3 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:55.255391 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1197/2381] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:49:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:49:55.525552 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08ba17-1bf5-4520-9302-96d908ac03de None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:55.527127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-8a08ba17-1bf5-4520-9302-96d908ac03de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:55.541730 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08ba17-1bf5-4520-9302-96d908ac03de None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:55.546681 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeecb50-1e72-4f90-95af-f4ccd9e8b7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:55.548207 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeecb50-1e72-4f90-95af-f4ccd9e8b7c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:55.548597 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeecb50-1e72-4f90-95af-f4ccd9e8b7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:55.549227 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeecb50-1e72-4f90-95af-f4ccd9e8b7c6 admin admin] PATH_INFO: `/v3/projects/ab7b8900cde9447dbff76a7cd1a36673` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:55.551807 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeecb50-1e72-4f90-95af-f4ccd9e8b7c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab7b8900cde9447dbff76a7cd1a36673)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:55.554116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeecb50-1e72-4f90-95af-f4ccd9e8b7c6 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:55.603595 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-5aeecb50-1e72-4f90-95af-f4ccd9e8b7c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab7b8900cde9447dbff76a7cd1a36673', 'request_id': 'req-5aeecb50-1e72-4f90-95af-f4ccd9e8b7c6'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:55.609810 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-5aeecb50-1e72-4f90-95af-f4ccd9e8b7c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab7b8900cde9447dbff76a7cd1a36673', 'request_id': 'req-5aeecb50-1e72-4f90-95af-f4ccd9e8b7c6'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:55.618457 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-5aeecb50-1e72-4f90-95af-f4ccd9e8b7c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab7b8900cde9447dbff76a7cd1a36673', 'request_id': 'req-5aeecb50-1e72-4f90-95af-f4ccd9e8b7c6'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:55.630673 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-5aeecb50-1e72-4f90-95af-f4ccd9e8b7c6 admin admin] The token cache is being invalidate because project ab7b8900cde9447dbff76a7cd1a36673 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:49:55.630968 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-5aeecb50-1e72-4f90-95af-f4ccd9e8b7c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:55.632299 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-5aeecb50-1e72-4f90-95af-f4ccd9e8b7c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:49:55.634351 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1185/2382] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:49:55 2022] DELETE /identity/v3/projects/ab7b8900cde9447dbff76a7cd1a36673 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:49:55.639205 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3971b-e1a7-4eb3-b3cf-4e8914eebd6c None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:55.639523 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3971b-e1a7-4eb3-b3cf-4e8914eebd6c None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:55.639830 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3971b-e1a7-4eb3-b3cf-4e8914eebd6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:55.657911 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-8bf3971b-e1a7-4eb3-b3cf-4e8914eebd6c None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:49:55.659191 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-8bf3971b-e1a7-4eb3-b3cf-4e8914eebd6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:55.701673 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:49:55.729458 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1198/2383] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:49:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:49:55.738565 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a678638-6402-4285-adb7-1e3a87643d1b None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:55.738883 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a678638-6402-4285-adb7-1e3a87643d1b None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:55.739200 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a678638-6402-4285-adb7-1e3a87643d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:55.758302 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-5a678638-6402-4285-adb7-1e3a87643d1b None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:49:55.759543 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-5a678638-6402-4285-adb7-1e3a87643d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:55.767412 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1186/2384] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:49:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:49:55.772088 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf3971b-e1a7-4eb3-b3cf-4e8914eebd6c None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:55.781863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf3971b-e1a7-4eb3-b3cf-4e8914eebd6c None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:55.787011 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a5784b-1e0a-494e-8e17-8b0ded72db7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:55.787657 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5784b-1e0a-494e-8e17-8b0ded72db7a admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:55.787940 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5784b-1e0a-494e-8e17-8b0ded72db7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:55.788215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5784b-1e0a-494e-8e17-8b0ded72db7a admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:55.788935 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a5784b-1e0a-494e-8e17-8b0ded72db7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:49:55.789268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a5784b-1e0a-494e-8e17-8b0ded72db7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:55.790842 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a5784b-1e0a-494e-8e17-8b0ded72db7a admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:55.803300 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1199/2385] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:49:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:49:55.807499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a678638-6402-4285-adb7-1e3a87643d1b None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:55.817135 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a678638-6402-4285-adb7-1e3a87643d1b None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:55.822007 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f0b59-f62b-49a4-b7fb-40f36009a437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:55.822705 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f0b59-f62b-49a4-b7fb-40f36009a437 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:55.823018 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f0b59-f62b-49a4-b7fb-40f36009a437 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:55.823321 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f0b59-f62b-49a4-b7fb-40f36009a437 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:55.824114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891f0b59-f62b-49a4-b7fb-40f36009a437 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:55.826270 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891f0b59-f62b-49a4-b7fb-40f36009a437 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:55.852301 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1187/2386] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:49:55 2022] POST /identity/v3/projects => generated 397 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:49:55.856539 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a5784b-1e0a-494e-8e17-8b0ded72db7a admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:55.867358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a5784b-1e0a-494e-8e17-8b0ded72db7a admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:55.923573 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9757f97-c9f6-473d-8d8d-119001dc8fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:55.924241 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9757f97-c9f6-473d-8d8d-119001dc8fae admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:55.924524 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9757f97-c9f6-473d-8d8d-119001dc8fae admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:55.924799 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9757f97-c9f6-473d-8d8d-119001dc8fae admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:55.926211 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9757f97-c9f6-473d-8d8d-119001dc8fae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:55.928115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9757f97-c9f6-473d-8d8d-119001dc8fae admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:55.947760 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1200/2387] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:49:55 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:49:55.952085 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f0b59-f62b-49a4-b7fb-40f36009a437 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:55.963358 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f0b59-f62b-49a4-b7fb-40f36009a437 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:55.968173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a105e3a-7832-41c7-be7a-a33bef6a2cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:55.968882 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a105e3a-7832-41c7-be7a-a33bef6a2cb0 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:55.969203 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a105e3a-7832-41c7-be7a-a33bef6a2cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:55.969480 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a105e3a-7832-41c7-be7a-a33bef6a2cb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:55.970200 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a105e3a-7832-41c7-be7a-a33bef6a2cb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:55.971853 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a105e3a-7832-41c7-be7a-a33bef6a2cb0 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:55.987096 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1188/2388] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:49:55.988919 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9757f97-c9f6-473d-8d8d-119001dc8fae admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:55.998957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9757f97-c9f6-473d-8d8d-119001dc8fae admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:56.004025 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126e971-6c85-474d-8169-a81f500f0d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:56.004681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126e971-6c85-474d-8169-a81f500f0d39 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:56.004968 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126e971-6c85-474d-8169-a81f500f0d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:56.005284 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126e971-6c85-474d-8169-a81f500f0d39 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:56.006041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126e971-6c85-474d-8169-a81f500f0d39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:56.007630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126e971-6c85-474d-8169-a81f500f0d39 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:56.017431 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1201/2389] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:49:56.021977 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a105e3a-7832-41c7-be7a-a33bef6a2cb0 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:56.031821 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a105e3a-7832-41c7-be7a-a33bef6a2cb0 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:56.037581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e1d8a-3ff6-4e79-b0de-9975b0a604fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:56.038457 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e1d8a-3ff6-4e79-b0de-9975b0a604fa admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:56.038770 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e1d8a-3ff6-4e79-b0de-9975b0a604fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:56.039047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e1d8a-3ff6-4e79-b0de-9975b0a604fa admin admin] PATH_INFO: `/v3/projects/885e2327a8804693a3fdbf52efe84ff0/users/6b6ca595eda447c2a24fcf397959659e/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:56.041259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:49:56.056064 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8e1d8a-3ff6-4e79-b0de-9975b0a604fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=885e2327a8804693a3fdbf52efe84ff0, user_id=6b6ca595eda447c2a24fcf397959659e, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:56.058803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8e1d8a-3ff6-4e79-b0de-9975b0a604fa admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:56.071467 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1189/2390] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:49:56 2022] PUT /identity/v3/projects/885e2327a8804693a3fdbf52efe84ff0/users/6b6ca595eda447c2a24fcf397959659e/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:49:56.136705 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7839ca-13b4-499c-b57d-6e0d1de8d25f None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:56.136992 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7839ca-13b4-499c-b57d-6e0d1de8d25f None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:56.137302 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7839ca-13b4-499c-b57d-6e0d1de8d25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:56.164572 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-6a7839ca-13b4-499c-b57d-6e0d1de8d25f None None] MFA Rules not processed for user `6b6ca595eda447c2a24fcf397959659e`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:49:56.165788 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-6a7839ca-13b4-499c-b57d-6e0d1de8d25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:56.227884 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1202/2391] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:49:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:49:56.239197 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e1d8a-3ff6-4e79-b0de-9975b0a604fa admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:49:56.240917 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-0f8e1d8a-3ff6-4e79-b0de-9975b0a604fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:49:56.296830 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e1d8a-3ff6-4e79-b0de-9975b0a604fa admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:49:56.300988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378d1aa-5981-4dd3-9220-fd585aa78e9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:49:56.301670 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378d1aa-5981-4dd3-9220-fd585aa78e9e service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:49:56.301955 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378d1aa-5981-4dd3-9220-fd585aa78e9e service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:49:56.302234 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378d1aa-5981-4dd3-9220-fd585aa78e9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:49:56.310758 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378d1aa-5981-4dd3-9220-fd585aa78e9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:49:56.312190 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378d1aa-5981-4dd3-9220-fd585aa78e9e service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:49:56.321459 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1190/2392] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:49:56 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:50:06.224757 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:50:06.317496 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:50:13.681432 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7839ca-13b4-499c-b57d-6e0d1de8d25f None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:50:13.744249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7839ca-13b4-499c-b57d-6e0d1de8d25f None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:50:13.748711 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4008b-317b-4f8c-b283-cbc77913b10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:50:13.749606 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4008b-317b-4f8c-b283-cbc77913b10d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:13.749889 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4008b-317b-4f8c-b283-cbc77913b10d admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:13.750168 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4008b-317b-4f8c-b283-cbc77913b10d admin admin] PATH_INFO: `/v3/users/6b6ca595eda447c2a24fcf397959659e` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:13.751345 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d4008b-317b-4f8c-b283-cbc77913b10d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b6ca595eda447c2a24fcf397959659e)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:50:13.753333 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d4008b-317b-4f8c-b283-cbc77913b10d admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:50:13.802618 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b2d4008b-317b-4f8c-b283-cbc77913b10d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b6ca595eda447c2a24fcf397959659e'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:50:13.809613 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b2d4008b-317b-4f8c-b283-cbc77913b10d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b6ca595eda447c2a24fcf397959659e'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:50:13.817068 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b2d4008b-317b-4f8c-b283-cbc77913b10d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b6ca595eda447c2a24fcf397959659e'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:50:13.817741 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b2d4008b-317b-4f8c-b283-cbc77913b10d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b6ca595eda447c2a24fcf397959659e'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:50:13.829597 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b2d4008b-317b-4f8c-b283-cbc77913b10d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b6ca595eda447c2a24fcf397959659e'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:50:13.835921 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1203/2393] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:50:13 2022] DELETE /identity/v3/users/6b6ca595eda447c2a24fcf397959659e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:50:13.840518 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be871b89-de4b-4cce-946f-654708ede4c1 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:13.841076 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be871b89-de4b-4cce-946f-654708ede4c1 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:13.841076 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be871b89-de4b-4cce-946f-654708ede4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:13.859228 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-be871b89-de4b-4cce-946f-654708ede4c1 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:50:13.860411 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-be871b89-de4b-4cce-946f-654708ede4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:50:13.919786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1191/2394] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:50:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:50:13.929619 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4008b-317b-4f8c-b283-cbc77913b10d admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:50:13.931030 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-b2d4008b-317b-4f8c-b283-cbc77913b10d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:50:13.987507 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4008b-317b-4f8c-b283-cbc77913b10d admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:50:13.991346 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4552c-8468-4bd3-966c-b9b5edd68c1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:50:13.991990 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4552c-8468-4bd3-966c-b9b5edd68c1e service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:13.992275 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4552c-8468-4bd3-966c-b9b5edd68c1e service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:13.992556 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4552c-8468-4bd3-966c-b9b5edd68c1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:14.001491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd4552c-8468-4bd3-966c-b9b5edd68c1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:50:14.002906 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd4552c-8468-4bd3-966c-b9b5edd68c1e service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:50:14.012490 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1204/2395] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:50:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:50:14.285707 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be871b89-de4b-4cce-946f-654708ede4c1 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:50:14.287167 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-be871b89-de4b-4cce-946f-654708ede4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:50:14.299473 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be871b89-de4b-4cce-946f-654708ede4c1 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:50:14.304635 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c274a-60cb-420f-be83-da32bbdbca67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:50:14.305560 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c274a-60cb-420f-be83-da32bbdbca67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:14.305853 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c274a-60cb-420f-be83-da32bbdbca67 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:14.306122 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c274a-60cb-420f-be83-da32bbdbca67 admin admin] PATH_INFO: `/v3/projects/885e2327a8804693a3fdbf52efe84ff0` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:14.307375 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260c274a-60cb-420f-be83-da32bbdbca67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=885e2327a8804693a3fdbf52efe84ff0)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:50:14.309307 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260c274a-60cb-420f-be83-da32bbdbca67 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:50:14.360637 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-260c274a-60cb-420f-be83-da32bbdbca67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '885e2327a8804693a3fdbf52efe84ff0', 'request_id': 'req-260c274a-60cb-420f-be83-da32bbdbca67'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:50:14.366931 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-260c274a-60cb-420f-be83-da32bbdbca67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '885e2327a8804693a3fdbf52efe84ff0', 'request_id': 'req-260c274a-60cb-420f-be83-da32bbdbca67'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:50:14.374078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-260c274a-60cb-420f-be83-da32bbdbca67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '885e2327a8804693a3fdbf52efe84ff0', 'request_id': 'req-260c274a-60cb-420f-be83-da32bbdbca67'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:50:14.385784 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-260c274a-60cb-420f-be83-da32bbdbca67 admin admin] The token cache is being invalidate because project 885e2327a8804693a3fdbf52efe84ff0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:50:14.386073 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-260c274a-60cb-420f-be83-da32bbdbca67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:50:14.386695 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-260c274a-60cb-420f-be83-da32bbdbca67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:50:14.388274 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1192/2396] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:50:14 2022] DELETE /identity/v3/projects/885e2327a8804693a3fdbf52efe84ff0 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:50:14.569381 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4552c-8468-4bd3-966c-b9b5edd68c1e service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:50:14.570890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-edd4552c-8468-4bd3-966c-b9b5edd68c1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:50:14.584611 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4552c-8468-4bd3-966c-b9b5edd68c1e service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:50:14.602747 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:50:14.645673 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc5326-c10e-4d28-ab9d-b8d43b59e2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:50:14.646557 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc5326-c10e-4d28-ab9d-b8d43b59e2c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:14.646838 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc5326-c10e-4d28-ab9d-b8d43b59e2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:14.647110 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc5326-c10e-4d28-ab9d-b8d43b59e2c5 admin admin] PATH_INFO: `/v3/users/8d6f31a080154a8dbdfda23b1c890767` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:14.648313 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fc5326-c10e-4d28-ab9d-b8d43b59e2c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d6f31a080154a8dbdfda23b1c890767)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:50:14.650252 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fc5326-c10e-4d28-ab9d-b8d43b59e2c5 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:50:14.697232 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-21fc5326-c10e-4d28-ab9d-b8d43b59e2c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d6f31a080154a8dbdfda23b1c890767'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:50:14.704557 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-21fc5326-c10e-4d28-ab9d-b8d43b59e2c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d6f31a080154a8dbdfda23b1c890767'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:50:14.711439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-21fc5326-c10e-4d28-ab9d-b8d43b59e2c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d6f31a080154a8dbdfda23b1c890767'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:50:14.712053 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-21fc5326-c10e-4d28-ab9d-b8d43b59e2c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d6f31a080154a8dbdfda23b1c890767'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:50:14.723946 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-21fc5326-c10e-4d28-ab9d-b8d43b59e2c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d6f31a080154a8dbdfda23b1c890767'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:50:14.730149 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1205/2397] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:50:14 2022] DELETE /identity/v3/users/8d6f31a080154a8dbdfda23b1c890767 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:50:14.734604 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d8938-6f14-48ab-a8e9-52503496317e None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:14.734899 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d8938-6f14-48ab-a8e9-52503496317e None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:14.735182 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d8938-6f14-48ab-a8e9-52503496317e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:14.755400 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-a06d8938-6f14-48ab-a8e9-52503496317e None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:50:14.757255 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a06d8938-6f14-48ab-a8e9-52503496317e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:50:14.820087 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1193/2398] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:50:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:50:14.829889 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc5326-c10e-4d28-ab9d-b8d43b59e2c5 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:50:14.831270 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-21fc5326-c10e-4d28-ab9d-b8d43b59e2c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:50:14.887283 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc5326-c10e-4d28-ab9d-b8d43b59e2c5 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:50:14.891062 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce47510-a672-4ec9-b473-7968356e0359 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:50:14.891709 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce47510-a672-4ec9-b473-7968356e0359 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:14.891990 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce47510-a672-4ec9-b473-7968356e0359 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:14.892265 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce47510-a672-4ec9-b473-7968356e0359 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:14.901278 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce47510-a672-4ec9-b473-7968356e0359 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:50:14.902695 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce47510-a672-4ec9-b473-7968356e0359 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:50:14.911989 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1206/2399] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:50:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:50:15.177024 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d8938-6f14-48ab-a8e9-52503496317e None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:50:15.178611 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a06d8938-6f14-48ab-a8e9-52503496317e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:50:15.190677 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d8938-6f14-48ab-a8e9-52503496317e None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:50:15.195593 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a002290-a955-4679-a092-9187c20be121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:50:15.196462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a002290-a955-4679-a092-9187c20be121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:15.196742 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a002290-a955-4679-a092-9187c20be121 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:15.197032 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a002290-a955-4679-a092-9187c20be121 admin admin] PATH_INFO: `/v3/projects/0f986cf8068f483c84f4061d69dbf1fa` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:15.198213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a002290-a955-4679-a092-9187c20be121 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f986cf8068f483c84f4061d69dbf1fa)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:50:15.200122 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a002290-a955-4679-a092-9187c20be121 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:50:15.223172 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:50:15.251375 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-0a002290-a955-4679-a092-9187c20be121 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f986cf8068f483c84f4061d69dbf1fa', 'request_id': 'req-0a002290-a955-4679-a092-9187c20be121'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:50:15.257511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-0a002290-a955-4679-a092-9187c20be121 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f986cf8068f483c84f4061d69dbf1fa', 'request_id': 'req-0a002290-a955-4679-a092-9187c20be121'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:50:15.264724 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-0a002290-a955-4679-a092-9187c20be121 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f986cf8068f483c84f4061d69dbf1fa', 'request_id': 'req-0a002290-a955-4679-a092-9187c20be121'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:50:15.274719 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-0a002290-a955-4679-a092-9187c20be121 admin admin] The token cache is being invalidate because project 0f986cf8068f483c84f4061d69dbf1fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:50:15.275006 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-0a002290-a955-4679-a092-9187c20be121 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:50:15.275614 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-0a002290-a955-4679-a092-9187c20be121 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:50:15.277168 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1194/2400] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:50:15 2022] DELETE /identity/v3/projects/0f986cf8068f483c84f4061d69dbf1fa => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:50:15.282446 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3e858-87d4-4528-8371-b89caf06849e None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:15.282739 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3e858-87d4-4528-8371-b89caf06849e None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:15.283016 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3e858-87d4-4528-8371-b89caf06849e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:15.302687 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-0cf3e858-87d4-4528-8371-b89caf06849e None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:50:15.303891 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-0cf3e858-87d4-4528-8371-b89caf06849e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:50:15.344093 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:50:15.364572 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1207/2401] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:50:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:50:15.372481 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c9a75-3b8a-4bc7-978e-dff6f0156c6e None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:15.372772 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c9a75-3b8a-4bc7-978e-dff6f0156c6e None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:15.373081 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c9a75-3b8a-4bc7-978e-dff6f0156c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:15.390892 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-408c9a75-3b8a-4bc7-978e-dff6f0156c6e None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:50:15.392116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-408c9a75-3b8a-4bc7-978e-dff6f0156c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:50:15.400712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1195/2402] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:50:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:50:15.404749 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3e858-87d4-4528-8371-b89caf06849e None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:50:15.414788 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3e858-87d4-4528-8371-b89caf06849e None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:50:15.419315 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e475d47-73da-4162-9447-0451f2f03556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:50:15.419962 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e475d47-73da-4162-9447-0451f2f03556 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:15.420247 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e475d47-73da-4162-9447-0451f2f03556 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:15.420524 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e475d47-73da-4162-9447-0451f2f03556 admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:15.421285 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e475d47-73da-4162-9447-0451f2f03556 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:50:15.421591 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e475d47-73da-4162-9447-0451f2f03556 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:50:15.423169 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e475d47-73da-4162-9447-0451f2f03556 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:50:15.434844 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1208/2403] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:50:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:50:15.438954 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c9a75-3b8a-4bc7-978e-dff6f0156c6e None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:50:15.448282 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c9a75-3b8a-4bc7-978e-dff6f0156c6e None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:50:15.452994 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438db5d9-a315-4f42-8ed5-edb7a9236cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:50:15.453671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438db5d9-a315-4f42-8ed5-edb7a9236cc0 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:15.453953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438db5d9-a315-4f42-8ed5-edb7a9236cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:15.454231 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438db5d9-a315-4f42-8ed5-edb7a9236cc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:15.454916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438db5d9-a315-4f42-8ed5-edb7a9236cc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:50:15.457451 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438db5d9-a315-4f42-8ed5-edb7a9236cc0 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:50:15.483860 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1196/2404] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:50:15 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:50:15.488051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e475d47-73da-4162-9447-0451f2f03556 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:50:15.498923 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e475d47-73da-4162-9447-0451f2f03556 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:50:15.555124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146f2d7-4e9c-45db-a3b0-089a620fb91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:50:15.555797 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146f2d7-4e9c-45db-a3b0-089a620fb91d admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:15.556086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146f2d7-4e9c-45db-a3b0-089a620fb91d admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:15.556370 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146f2d7-4e9c-45db-a3b0-089a620fb91d admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:15.557125 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9146f2d7-4e9c-45db-a3b0-089a620fb91d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:50:15.559072 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9146f2d7-4e9c-45db-a3b0-089a620fb91d admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:50:15.578572 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1209/2405] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:50:15 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:50:15.583494 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438db5d9-a315-4f42-8ed5-edb7a9236cc0 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:50:15.593770 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438db5d9-a315-4f42-8ed5-edb7a9236cc0 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:50:15.599118 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0cf373-bf7f-4701-a45d-cdd892c1001c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:50:15.599796 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0cf373-bf7f-4701-a45d-cdd892c1001c admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:15.600080 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0cf373-bf7f-4701-a45d-cdd892c1001c admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:15.600358 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0cf373-bf7f-4701-a45d-cdd892c1001c admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:15.601144 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0cf373-bf7f-4701-a45d-cdd892c1001c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:50:15.602765 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0cf373-bf7f-4701-a45d-cdd892c1001c admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:50:15.613148 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1197/2406] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:50:15.616964 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146f2d7-4e9c-45db-a3b0-089a620fb91d admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:50:15.626634 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146f2d7-4e9c-45db-a3b0-089a620fb91d admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:50:15.631270 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f90ddc-656b-48ed-9647-fa32a03c9c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:50:15.631921 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f90ddc-656b-48ed-9647-fa32a03c9c10 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:15.632204 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f90ddc-656b-48ed-9647-fa32a03c9c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:15.632482 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f90ddc-656b-48ed-9647-fa32a03c9c10 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:15.633239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f90ddc-656b-48ed-9647-fa32a03c9c10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:50:15.634810 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f90ddc-656b-48ed-9647-fa32a03c9c10 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:50:15.645179 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1210/2407] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:50:15.648980 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0cf373-bf7f-4701-a45d-cdd892c1001c admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:50:15.659063 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0cf373-bf7f-4701-a45d-cdd892c1001c admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:50:15.664247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98928d8-c677-4260-9338-f62b6be31f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:50:15.665150 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98928d8-c677-4260-9338-f62b6be31f0d admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:15.665440 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98928d8-c677-4260-9338-f62b6be31f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:15.665720 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98928d8-c677-4260-9338-f62b6be31f0d admin admin] PATH_INFO: `/v3/projects/7bf864bb1764452e8e3a6f0d5929da9f/users/32a8191b1c184c75b428579ace2da61c/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:15.680529 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98928d8-c677-4260-9338-f62b6be31f0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bf864bb1764452e8e3a6f0d5929da9f, user_id=32a8191b1c184c75b428579ace2da61c, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:50:15.683243 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98928d8-c677-4260-9338-f62b6be31f0d admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:50:15.697484 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1198/2408] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:50:15 2022] PUT /identity/v3/projects/7bf864bb1764452e8e3a6f0d5929da9f/users/32a8191b1c184c75b428579ace2da61c/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:50:15.702458 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4901603-b2d9-4068-b6ca-1b7971b40fd1 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:15.702936 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4901603-b2d9-4068-b6ca-1b7971b40fd1 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:15.703050 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4901603-b2d9-4068-b6ca-1b7971b40fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:15.721093 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-a4901603-b2d9-4068-b6ca-1b7971b40fd1 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:50:15.722313 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-a4901603-b2d9-4068-b6ca-1b7971b40fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:50:15.780016 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1211/2409] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:50:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:50:15.789800 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98928d8-c677-4260-9338-f62b6be31f0d admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:50:15.791468 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a98928d8-c677-4260-9338-f62b6be31f0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:50:15.851168 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98928d8-c677-4260-9338-f62b6be31f0d admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:50:15.854797 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efaa0ea-d187-4b81-8046-8d9ff779f57d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:50:15.855446 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efaa0ea-d187-4b81-8046-8d9ff779f57d service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:15.855730 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efaa0ea-d187-4b81-8046-8d9ff779f57d service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:15.856009 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efaa0ea-d187-4b81-8046-8d9ff779f57d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:15.865203 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efaa0ea-d187-4b81-8046-8d9ff779f57d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:50:15.866658 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efaa0ea-d187-4b81-8046-8d9ff779f57d service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:50:15.876009 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1199/2410] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:50:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:50:19.831382 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8e6a9-58ab-4306-8aee-7f5239e5cda8 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:19.832738 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8e6a9-58ab-4306-8aee-7f5239e5cda8 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:19.832931 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8e6a9-58ab-4306-8aee-7f5239e5cda8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:19.862336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-4dd8e6a9-58ab-4306-8aee-7f5239e5cda8 None None] MFA Rules not processed for user `32a8191b1c184c75b428579ace2da61c`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:50:19.864566 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-4dd8e6a9-58ab-4306-8aee-7f5239e5cda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:50:19.912037 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:50:19.945783 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1212/2411] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:50:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:50:19.950080 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efaa0ea-d187-4b81-8046-8d9ff779f57d service neutron] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:50:19.962041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efaa0ea-d187-4b81-8046-8d9ff779f57d service neutron] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:50:19.967441 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5177d1-b520-4dbd-9e9d-fb57c10f2fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:50:19.968123 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5177d1-b520-4dbd-9e9d-fb57c10f2fbe admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:19.968403 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5177d1-b520-4dbd-9e9d-fb57c10f2fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:19.969064 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5177d1-b520-4dbd-9e9d-fb57c10f2fbe admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:19.969784 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5177d1-b520-4dbd-9e9d-fb57c10f2fbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:50:19.971734 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5177d1-b520-4dbd-9e9d-fb57c10f2fbe admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:50:20.012933 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1200/2412] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:50:19 2022] POST /identity/v3/projects => generated 379 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:50:20.016573 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd8e6a9-58ab-4306-8aee-7f5239e5cda8 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:50:20.029132 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd8e6a9-58ab-4306-8aee-7f5239e5cda8 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:50:20.101588 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966bba8-9458-4ce1-b740-4ed9996c90dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:50:20.102545 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966bba8-9458-4ce1-b740-4ed9996c90dc admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:20.102861 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966bba8-9458-4ce1-b740-4ed9996c90dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:20.103256 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966bba8-9458-4ce1-b740-4ed9996c90dc admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:20.104003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d966bba8-9458-4ce1-b740-4ed9996c90dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:50:20.105978 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d966bba8-9458-4ce1-b740-4ed9996c90dc admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:50:20.128811 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1213/2413] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:50:20 2022] POST /identity/v3/users => generated 296 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:50:20.132800 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5177d1-b520-4dbd-9e9d-fb57c10f2fbe admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:50:20.143651 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5177d1-b520-4dbd-9e9d-fb57c10f2fbe admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:50:20.151119 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc4dbb-50f7-4c69-a081-95503b498a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:50:20.151800 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc4dbb-50f7-4c69-a081-95503b498a36 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:20.152081 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc4dbb-50f7-4c69-a081-95503b498a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:20.152362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc4dbb-50f7-4c69-a081-95503b498a36 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:20.153120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dc4dbb-50f7-4c69-a081-95503b498a36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:50:20.154732 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dc4dbb-50f7-4c69-a081-95503b498a36 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:50:20.168201 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1201/2414] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:50:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:50:20.172171 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966bba8-9458-4ce1-b740-4ed9996c90dc admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:50:20.192563 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966bba8-9458-4ce1-b740-4ed9996c90dc admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:50:20.221657 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579164c-c20d-4d90-9da1-1d7e4206c087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:50:20.223472 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579164c-c20d-4d90-9da1-1d7e4206c087 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:20.223749 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579164c-c20d-4d90-9da1-1d7e4206c087 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:20.224024 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579164c-c20d-4d90-9da1-1d7e4206c087 admin admin] PATH_INFO: `/v3/projects/ca371b04b33244adae871d37770223d9/users/83c2308390c64052984b7150f85838a7/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:20.240221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579164c-c20d-4d90-9da1-1d7e4206c087 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca371b04b33244adae871d37770223d9, user_id=83c2308390c64052984b7150f85838a7, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:50:20.242914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579164c-c20d-4d90-9da1-1d7e4206c087 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:50:20.262871 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1214/2415] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:50:20 2022] PUT /identity/v3/projects/ca371b04b33244adae871d37770223d9/users/83c2308390c64052984b7150f85838a7/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:50:20.266926 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc4dbb-50f7-4c69-a081-95503b498a36 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:50:20.278114 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc4dbb-50f7-4c69-a081-95503b498a36 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:50:20.295312 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:50:20.359291 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711759a0-14fb-42d4-841e-6a5165788064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:50:20.359976 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711759a0-14fb-42d4-841e-6a5165788064 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:20.360257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711759a0-14fb-42d4-841e-6a5165788064 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:20.360558 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711759a0-14fb-42d4-841e-6a5165788064 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:20.361375 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711759a0-14fb-42d4-841e-6a5165788064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:50:20.363641 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711759a0-14fb-42d4-841e-6a5165788064 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:50:20.374637 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1202/2416] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:50:20 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:50:20.379524 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579164c-c20d-4d90-9da1-1d7e4206c087 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:50:20.391286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579164c-c20d-4d90-9da1-1d7e4206c087 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:50:20.396336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6a7bc-d4a5-49d4-9484-69204e8fe5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:50:20.397389 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6a7bc-d4a5-49d4-9484-69204e8fe5e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:20.397737 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6a7bc-d4a5-49d4-9484-69204e8fe5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:20.398119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6a7bc-d4a5-49d4-9484-69204e8fe5e4 admin admin] PATH_INFO: `/v3/projects/ca371b04b33244adae871d37770223d9/users/83c2308390c64052984b7150f85838a7/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:20.400334 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad6a7bc-d4a5-49d4-9484-69204e8fe5e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca371b04b33244adae871d37770223d9, user_id=83c2308390c64052984b7150f85838a7, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:50:20.402987 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad6a7bc-d4a5-49d4-9484-69204e8fe5e4 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:50:20.415449 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1215/2417] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:50:20 2022] PUT /identity/v3/projects/ca371b04b33244adae871d37770223d9/users/83c2308390c64052984b7150f85838a7/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:50:24.605555 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81caa561-f352-4939-9f69-ce72c011d5fc None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:24.606148 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81caa561-f352-4939-9f69-ce72c011d5fc None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:24.606148 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81caa561-f352-4939-9f69-ce72c011d5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:24.633521 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-81caa561-f352-4939-9f69-ce72c011d5fc None None] MFA Rules not processed for user `83c2308390c64052984b7150f85838a7`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:50:24.634809 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-81caa561-f352-4939-9f69-ce72c011d5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:50:24.730596 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1203/2418] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:50:24 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:50:24.748945 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6a7bc-d4a5-49d4-9484-69204e8fe5e4 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:50:24.751034 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-2ad6a7bc-d4a5-49d4-9484-69204e8fe5e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:50:24.848687 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6a7bc-d4a5-49d4-9484-69204e8fe5e4 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:50:24.854214 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf19026-2162-49e8-8d65-8f912de69619 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:50:24.854883 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf19026-2162-49e8-8d65-8f912de69619 service nova] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:24.855434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf19026-2162-49e8-8d65-8f912de69619 service nova] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:24.855434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf19026-2162-49e8-8d65-8f912de69619 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:24.864637 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:50:24.914088 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf19026-2162-49e8-8d65-8f912de69619 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:50:24.915697 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf19026-2162-49e8-8d65-8f912de69619 service nova] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:50:24.924505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1216/2419] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:50:24 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:50:34.726271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=57 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:50:34.920530 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=23 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:50:50.739217 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81caa561-f352-4939-9f69-ce72c011d5fc None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:50:50.740716 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-81caa561-f352-4939-9f69-ce72c011d5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:50:50.795880 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81caa561-f352-4939-9f69-ce72c011d5fc None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:50:50.800171 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b7dc5a-854e-4b4b-8c3e-7046fc07a38e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:50:50.800811 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7dc5a-854e-4b4b-8c3e-7046fc07a38e service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:50.801115 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7dc5a-854e-4b4b-8c3e-7046fc07a38e service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:50.801387 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7dc5a-854e-4b4b-8c3e-7046fc07a38e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:50.803443 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-27b7dc5a-854e-4b4b-8c3e-7046fc07a38e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:50:50.817942 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b7dc5a-854e-4b4b-8c3e-7046fc07a38e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:50:50.819372 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b7dc5a-854e-4b4b-8c3e-7046fc07a38e service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:50:50.830096 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1204/2420] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:50:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:50:51.687252 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4cea89-1b26-4509-9c9c-45e1472e48d4 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:51.687787 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4cea89-1b26-4509-9c9c-45e1472e48d4 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:51.687787 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4cea89-1b26-4509-9c9c-45e1472e48d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:51.705114 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-fb4cea89-1b26-4509-9c9c-45e1472e48d4 None None] MFA Rules not processed for user `32a8191b1c184c75b428579ace2da61c`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:50:51.706298 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-fb4cea89-1b26-4509-9c9c-45e1472e48d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:50:51.712976 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1217/2421] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:50:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:50:51.749393 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b7dc5a-854e-4b4b-8c3e-7046fc07a38e service neutron] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:50:51.750826 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-27b7dc5a-854e-4b4b-8c3e-7046fc07a38e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:50:51.806527 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b7dc5a-854e-4b4b-8c3e-7046fc07a38e service neutron] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:50:51.810136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19365c81-6075-4c37-88b0-6dd6999cc79d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:50:51.810779 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19365c81-6075-4c37-88b0-6dd6999cc79d service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:51.811062 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19365c81-6075-4c37-88b0-6dd6999cc79d service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:51.811339 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19365c81-6075-4c37-88b0-6dd6999cc79d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:51.813091 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-19365c81-6075-4c37-88b0-6dd6999cc79d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:50:51.826204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19365c81-6075-4c37-88b0-6dd6999cc79d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:50:51.827614 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19365c81-6075-4c37-88b0-6dd6999cc79d service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:50:51.838703 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1205/2422] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:50:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:50:52.125237 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4cea89-1b26-4509-9c9c-45e1472e48d4 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:50:52.140134 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4cea89-1b26-4509-9c9c-45e1472e48d4 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:50:52.144210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355856fd-fa93-418b-973e-057ebf6a383a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:50:52.144851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355856fd-fa93-418b-973e-057ebf6a383a service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:52.145168 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355856fd-fa93-418b-973e-057ebf6a383a service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:52.145442 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355856fd-fa93-418b-973e-057ebf6a383a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:52.147163 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-355856fd-fa93-418b-973e-057ebf6a383a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:50:52.156318 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355856fd-fa93-418b-973e-057ebf6a383a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:50:52.157778 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355856fd-fa93-418b-973e-057ebf6a383a service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:50:52.166172 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1218/2423] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:50:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:50:52.600588 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19365c81-6075-4c37-88b0-6dd6999cc79d service cinder] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:50:52.601973 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-19365c81-6075-4c37-88b0-6dd6999cc79d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:50:52.671862 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19365c81-6075-4c37-88b0-6dd6999cc79d service cinder] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:50:52.677148 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d135550c-1a94-4a26-998d-eb58b94ce48f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '890d2ae150d647a18dae25159cf97484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:50:52.677793 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135550c-1a94-4a26-998d-eb58b94ce48f service placement] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:52.678074 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135550c-1a94-4a26-998d-eb58b94ce48f service placement] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:52.678349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135550c-1a94-4a26-998d-eb58b94ce48f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:52.680083 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-d135550c-1a94-4a26-998d-eb58b94ce48f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:50:52.691744 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d135550c-1a94-4a26-998d-eb58b94ce48f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:50:52.693340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d135550c-1a94-4a26-998d-eb58b94ce48f service placement] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:50:52.702902 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1206/2424] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:50:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:50:54.680277 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355856fd-fa93-418b-973e-057ebf6a383a service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:50:54.694450 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355856fd-fa93-418b-973e-057ebf6a383a service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:50:54.698387 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa2a57-5d92-4de8-9bef-3e5c0865dafd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:50:54.699121 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa2a57-5d92-4de8-9bef-3e5c0865dafd service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:54.699475 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa2a57-5d92-4de8-9bef-3e5c0865dafd service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:54.699822 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa2a57-5d92-4de8-9bef-3e5c0865dafd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:54.701728 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-90aa2a57-5d92-4de8-9bef-3e5c0865dafd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:50:54.713862 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa2a57-5d92-4de8-9bef-3e5c0865dafd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:50:54.715385 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa2a57-5d92-4de8-9bef-3e5c0865dafd service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:50:54.725441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1219/2425] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:50:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:50:55.572609 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d135550c-1a94-4a26-998d-eb58b94ce48f service placement] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:50:55.618577 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d135550c-1a94-4a26-998d-eb58b94ce48f service placement] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:50:55.626252 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767b8c11-6cb3-488f-9a39-bc0aef8079f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:50:55.627456 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b8c11-6cb3-488f-9a39-bc0aef8079f3 service nova] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:50:55.627909 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b8c11-6cb3-488f-9a39-bc0aef8079f3 service nova] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:50:55.628455 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b8c11-6cb3-488f-9a39-bc0aef8079f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:50:55.630328 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-767b8c11-6cb3-488f-9a39-bc0aef8079f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:50:55.644969 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767b8c11-6cb3-488f-9a39-bc0aef8079f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:50:55.646899 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767b8c11-6cb3-488f-9a39-bc0aef8079f3 service nova] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:50:55.658959 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1207/2426] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:50:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:51:00.822784 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa2a57-5d92-4de8-9bef-3e5c0865dafd service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:51:00.824256 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-90aa2a57-5d92-4de8-9bef-3e5c0865dafd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:51:00.841282 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa2a57-5d92-4de8-9bef-3e5c0865dafd service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:51:00.848129 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a6b05-5850-4d23-a5fa-005f0351cc3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:51:00.848805 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a6b05-5850-4d23-a5fa-005f0351cc3a service nova] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:51:00.849122 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a6b05-5850-4d23-a5fa-005f0351cc3a service nova] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:51:00.849402 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a6b05-5850-4d23-a5fa-005f0351cc3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:51:00.851667 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-123a6b05-5850-4d23-a5fa-005f0351cc3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:51:00.927665 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a6b05-5850-4d23-a5fa-005f0351cc3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:51:00.930019 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a6b05-5850-4d23-a5fa-005f0351cc3a service nova] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:51:00.940037 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1220/2427] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:51:00 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:51:05.653793 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=90 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:51:10.935526 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=43 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:51:11.198121 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3c468-a14f-4cbc-a555-bda85826114b None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:51:11.198694 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3c468-a14f-4cbc-a555-bda85826114b None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:51:11.199035 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3c468-a14f-4cbc-a555-bda85826114b None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:51:11.200892 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1208/2428] 10.222.0.20 () {54 vars in 921 bytes} [Wed Jul 6 13:51:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:51:11.208811 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42818e6c-2130-4e49-9f43-feed40e9ded7 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:51:11.209145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42818e6c-2130-4e49-9f43-feed40e9ded7 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:51:11.209423 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42818e6c-2130-4e49-9f43-feed40e9ded7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:51:11.227909 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-42818e6c-2130-4e49-9f43-feed40e9ded7 None None] MFA Rules not processed for user `b9f250f5dbc54d2e9293962ece848f6c`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:51:11.229460 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-42818e6c-2130-4e49-9f43-feed40e9ded7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:51:11.236207 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1221/2429] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Jul 6 13:51:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:51:11.251428 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3c468-a14f-4cbc-a555-bda85826114b None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:51:11.263863 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3c468-a14f-4cbc-a555-bda85826114b None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:51:11.268050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0250fd-c7d8-45fc-a2f9-7008d22bd9b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:51:11.269408 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0250fd-c7d8-45fc-a2f9-7008d22bd9b8 service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:51:11.269764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0250fd-c7d8-45fc-a2f9-7008d22bd9b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:51:11.270312 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0250fd-c7d8-45fc-a2f9-7008d22bd9b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:51:11.278239 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0250fd-c7d8-45fc-a2f9-7008d22bd9b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:51:11.279738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0250fd-c7d8-45fc-a2f9-7008d22bd9b8 service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:51:11.290324 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1209/2430] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:51:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:51:11.459244 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda4cf9-7268-4bb9-a2de-64b72a96cc1f None None] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:51:11.459519 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda4cf9-7268-4bb9-a2de-64b72a96cc1f None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:51:11.459890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda4cf9-7268-4bb9-a2de-64b72a96cc1f None None] PATH_INFO: `/` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:51:11.461422 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1222/2431] 10.222.0.20 () {54 vars in 921 bytes} [Wed Jul 6 13:51:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:51:11.467314 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252902bf-bc98-4907-af1a-959a43612317 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:51:11.467825 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252902bf-bc98-4907-af1a-959a43612317 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:51:11.467946 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252902bf-bc98-4907-af1a-959a43612317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:51:11.508395 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-252902bf-bc98-4907-af1a-959a43612317 None None] MFA Rules not processed for user `6647eb1e8a1544ef9d3a9806f2d1fc51`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:51:11.509648 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-252902bf-bc98-4907-af1a-959a43612317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:51:11.521311 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1210/2432] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Jul 6 13:51:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:51:11.529971 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda4cf9-7268-4bb9-a2de-64b72a96cc1f None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:51:11.540905 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda4cf9-7268-4bb9-a2de-64b72a96cc1f None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:51:11.544653 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bcbd6f-71f0-433b-95d9-aee98dc7db42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:51:11.545338 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcbd6f-71f0-433b-95d9-aee98dc7db42 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:51:11.545616 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcbd6f-71f0-433b-95d9-aee98dc7db42 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:51:11.545892 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcbd6f-71f0-433b-95d9-aee98dc7db42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:51:11.556900 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bcbd6f-71f0-433b-95d9-aee98dc7db42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:51:11.558420 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bcbd6f-71f0-433b-95d9-aee98dc7db42 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:51:11.570220 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1223/2433] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:51:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:51:12.922377 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252902bf-bc98-4907-af1a-959a43612317 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:51:12.932877 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252902bf-bc98-4907-af1a-959a43612317 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:51:12.936560 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2095bba8-b61d-4b38-bb3a-9489e19ce35b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:51:12.937263 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095bba8-b61d-4b38-bb3a-9489e19ce35b service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:51:12.937596 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095bba8-b61d-4b38-bb3a-9489e19ce35b service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:51:12.937870 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095bba8-b61d-4b38-bb3a-9489e19ce35b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:51:12.947305 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2095bba8-b61d-4b38-bb3a-9489e19ce35b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:51:12.948810 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2095bba8-b61d-4b38-bb3a-9489e19ce35b service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:51:12.960499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1211/2434] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:51:12 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:51:15.465829 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bcbd6f-71f0-433b-95d9-aee98dc7db42 service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:51:15.477803 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bcbd6f-71f0-433b-95d9-aee98dc7db42 service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:51:15.483581 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3effe1-5948-4c7b-b717-27f8c4aea694 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:51:15.484262 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3effe1-5948-4c7b-b717-27f8c4aea694 service nova] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:51:15.484552 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3effe1-5948-4c7b-b717-27f8c4aea694 service nova] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:51:15.484830 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3effe1-5948-4c7b-b717-27f8c4aea694 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:51:15.486599 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-0f3effe1-5948-4c7b-b717-27f8c4aea694 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:51:15.499189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3effe1-5948-4c7b-b717-27f8c4aea694 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:51:15.500752 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3effe1-5948-4c7b-b717-27f8c4aea694 service nova] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:51:15.512049 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1224/2435] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:51:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:51:22.955541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=21 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:51:25.507405 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=15 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:51:27.975115 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2095bba8-b61d-4b38-bb3a-9489e19ce35b service neutron] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:51:27.988079 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2095bba8-b61d-4b38-bb3a-9489e19ce35b service neutron] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:51:27.995167 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ca56e-a1c2-4f80-ab4c-902cd6d1e784 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:51:27.996044 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ca56e-a1c2-4f80-ab4c-902cd6d1e784 service nova] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:51:27.996331 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ca56e-a1c2-4f80-ab4c-902cd6d1e784 service nova] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:51:27.996608 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ca56e-a1c2-4f80-ab4c-902cd6d1e784 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:51:27.998592 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-2a4ca56e-a1c2-4f80-ab4c-902cd6d1e784 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:51:28.012836 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ca56e-a1c2-4f80-ab4c-902cd6d1e784 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:51:28.014449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ca56e-a1c2-4f80-ab4c-902cd6d1e784 service nova] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:51:28.031383 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1212/2436] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:51:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:51:30.310663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99064e-acbb-4637-8585-a5a7520b27de None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:51:30.311208 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99064e-acbb-4637-8585-a5a7520b27de None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:51:30.311208 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99064e-acbb-4637-8585-a5a7520b27de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:51:30.329839 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-ee99064e-acbb-4637-8585-a5a7520b27de None None] MFA Rules not processed for user `0c83c05bec4f456d8aa7d51732ddc8af`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:51:30.331064 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-ee99064e-acbb-4637-8585-a5a7520b27de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:51:30.338532 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1225/2437] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:51:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:51:30.609688 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ca56e-a1c2-4f80-ab4c-902cd6d1e784 service nova] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:51:30.622895 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ca56e-a1c2-4f80-ab4c-902cd6d1e784 service nova] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:51:30.628367 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd136ad5-5395-4f41-82a7-762687d3135a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:51:30.629036 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd136ad5-5395-4f41-82a7-762687d3135a service nova] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:51:30.629322 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd136ad5-5395-4f41-82a7-762687d3135a service nova] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:51:30.629643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd136ad5-5395-4f41-82a7-762687d3135a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:51:30.631491 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-bd136ad5-5395-4f41-82a7-762687d3135a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:51:30.692069 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd136ad5-5395-4f41-82a7-762687d3135a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:51:30.694013 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd136ad5-5395-4f41-82a7-762687d3135a service nova] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:51:30.703573 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1213/2438] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:51:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:51:33.309036 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99064e-acbb-4637-8585-a5a7520b27de None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:51:33.327222 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99064e-acbb-4637-8585-a5a7520b27de None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:51:33.333991 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01d0ed-f30a-4cea-9724-a314b0851a17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:51:33.335130 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01d0ed-f30a-4cea-9724-a314b0851a17 service nova] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:51:33.335653 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01d0ed-f30a-4cea-9724-a314b0851a17 service nova] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:51:33.336161 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01d0ed-f30a-4cea-9724-a314b0851a17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:51:33.338196 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-3c01d0ed-f30a-4cea-9724-a314b0851a17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:51:33.400768 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01d0ed-f30a-4cea-9724-a314b0851a17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:51:33.402424 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01d0ed-f30a-4cea-9724-a314b0851a17 service nova] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:51:33.411901 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1226/2439] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:51:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:51:33.751502 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd136ad5-5395-4f41-82a7-762687d3135a service nova] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:51:33.762647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd136ad5-5395-4f41-82a7-762687d3135a service nova] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:51:33.768216 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ddea3-3f8f-4c17-a4fc-2406118c4e3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:51:33.768855 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ddea3-3f8f-4c17-a4fc-2406118c4e3e service nova] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:51:33.769174 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ddea3-3f8f-4c17-a4fc-2406118c4e3e service nova] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:51:33.769455 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ddea3-3f8f-4c17-a4fc-2406118c4e3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:51:33.771213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-ea7ddea3-3f8f-4c17-a4fc-2406118c4e3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:51:33.829862 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ddea3-3f8f-4c17-a4fc-2406118c4e3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:51:33.831361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ddea3-3f8f-4c17-a4fc-2406118c4e3e service nova] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:51:33.840637 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1214/2440] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:51:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:51:43.407284 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=31 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:51:43.836481 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=62 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:51:49.476557 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc0371-1217-4ce5-bb85-59ae490a708c None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:51:49.477088 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc0371-1217-4ce5-bb85-59ae490a708c None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:51:49.477203 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc0371-1217-4ce5-bb85-59ae490a708c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:51:49.494331 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-2cdc0371-1217-4ce5-bb85-59ae490a708c None None] MFA Rules not processed for user `7b572a8f465048558fc969b85b2aec5b`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:51:49.495525 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-2cdc0371-1217-4ce5-bb85-59ae490a708c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:51:49.502542 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1227/2441] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:51:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:51:59.489558 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:52:37.290173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ddea3-3f8f-4c17-a4fc-2406118c4e3e service nova] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:52:37.301695 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ddea3-3f8f-4c17-a4fc-2406118c4e3e service nova] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:52:37.308043 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d3e16-4284-49df-89d0-5dc2306f680c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:52:37.308697 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d3e16-4284-49df-89d0-5dc2306f680c service nova] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:52:37.308980 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d3e16-4284-49df-89d0-5dc2306f680c service nova] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:52:37.309290 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d3e16-4284-49df-89d0-5dc2306f680c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:52:37.311065 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-4f7d3e16-4284-49df-89d0-5dc2306f680c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:52:37.376866 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d3e16-4284-49df-89d0-5dc2306f680c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:52:37.378484 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d3e16-4284-49df-89d0-5dc2306f680c service nova] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:52:37.388062 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1215/2442] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:52:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:52:37.477853 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f0725-6694-4f46-bcd0-c641b17930d8 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:52:37.478308 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f0725-6694-4f46-bcd0-c641b17930d8 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:52:37.478409 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f0725-6694-4f46-bcd0-c641b17930d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:52:37.525626 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-291f0725-6694-4f46-bcd0-c641b17930d8 None None] MFA Rules not processed for user `c4535134181e489fb532f617264d5855`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:52:37.526811 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-291f0725-6694-4f46-bcd0-c641b17930d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:52:37.589625 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1228/2443] 10.222.0.20 () {58 vars in 967 bytes} [Wed Jul 6 13:52:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:52:37.637928 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d3e16-4284-49df-89d0-5dc2306f680c service nova] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:52:37.639322 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-4f7d3e16-4284-49df-89d0-5dc2306f680c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:52:37.733645 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d3e16-4284-49df-89d0-5dc2306f680c service nova] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:52:37.738994 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed11a4d-636b-48d7-9ffd-681c1e9f8c0d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c1bea05c37646b0a05fa8dda0367ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:52:37.739667 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed11a4d-636b-48d7-9ffd-681c1e9f8c0d service swift] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:52:37.740081 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed11a4d-636b-48d7-9ffd-681c1e9f8c0d service swift] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:52:37.740360 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed11a4d-636b-48d7-9ffd-681c1e9f8c0d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:52:37.749051 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed11a4d-636b-48d7-9ffd-681c1e9f8c0d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:52:37.750660 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed11a4d-636b-48d7-9ffd-681c1e9f8c0d service swift] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:52:37.759545 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1216/2444] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jul 6 13:52:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 13:52:38.152851 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f0725-6694-4f46-bcd0-c641b17930d8 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:52:38.162033 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f0725-6694-4f46-bcd0-c641b17930d8 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:52:38.165601 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf713e13-2f5e-4663-af11-8d6c7a76548e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:52:38.166238 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf713e13-2f5e-4663-af11-8d6c7a76548e service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:52:38.166521 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf713e13-2f5e-4663-af11-8d6c7a76548e service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:52:38.166800 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf713e13-2f5e-4663-af11-8d6c7a76548e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:52:38.168546 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-cf713e13-2f5e-4663-af11-8d6c7a76548e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:52:38.231684 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf713e13-2f5e-4663-af11-8d6c7a76548e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:52:38.233145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf713e13-2f5e-4663-af11-8d6c7a76548e service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:52:38.243334 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1229/2445] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:52:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:52:41.658905 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed11a4d-636b-48d7-9ffd-681c1e9f8c0d service swift] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:52:41.670804 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed11a4d-636b-48d7-9ffd-681c1e9f8c0d service swift] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:52:41.676051 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1be521-3dae-43d3-a2c2-0533d639e3f8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '890d2ae150d647a18dae25159cf97484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:52:41.676694 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1be521-3dae-43d3-a2c2-0533d639e3f8 service placement] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:52:41.677197 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1be521-3dae-43d3-a2c2-0533d639e3f8 service placement] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:52:41.677656 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1be521-3dae-43d3-a2c2-0533d639e3f8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:52:41.680057 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-7a1be521-3dae-43d3-a2c2-0533d639e3f8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:52:41.693502 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1be521-3dae-43d3-a2c2-0533d639e3f8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:52:41.696276 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1be521-3dae-43d3-a2c2-0533d639e3f8 service placement] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:52:41.711733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1217/2446] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:52:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:52:44.849561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf713e13-2f5e-4663-af11-8d6c7a76548e service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:52:44.851153 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-cf713e13-2f5e-4663-af11-8d6c7a76548e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:52:44.866183 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf713e13-2f5e-4663-af11-8d6c7a76548e service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:52:44.871611 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa9954-b000-4039-a62b-fee54e2737b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:52:44.872484 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa9954-b000-4039-a62b-fee54e2737b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:52:44.872768 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa9954-b000-4039-a62b-fee54e2737b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:52:44.873145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa9954-b000-4039-a62b-fee54e2737b6 admin admin] PATH_INFO: `/v3/users/19ccda8f7ce447c490910ffa14165c8b` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:52:44.874353 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aa9954-b000-4039-a62b-fee54e2737b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19ccda8f7ce447c490910ffa14165c8b)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:52:44.876275 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aa9954-b000-4039-a62b-fee54e2737b6 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:52:44.924957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-03aa9954-b000-4039-a62b-fee54e2737b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19ccda8f7ce447c490910ffa14165c8b'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:52:44.932343 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-03aa9954-b000-4039-a62b-fee54e2737b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19ccda8f7ce447c490910ffa14165c8b'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:52:44.937885 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:52:44.940348 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-03aa9954-b000-4039-a62b-fee54e2737b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19ccda8f7ce447c490910ffa14165c8b'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:52:44.941247 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-03aa9954-b000-4039-a62b-fee54e2737b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19ccda8f7ce447c490910ffa14165c8b'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:52:44.955629 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-03aa9954-b000-4039-a62b-fee54e2737b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19ccda8f7ce447c490910ffa14165c8b'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:52:44.962163 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1230/2447] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:52:44 2022] DELETE /identity/v3/users/19ccda8f7ce447c490910ffa14165c8b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:52:45.142232 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1be521-3dae-43d3-a2c2-0533d639e3f8 service placement] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:52:45.155660 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1be521-3dae-43d3-a2c2-0533d639e3f8 service placement] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:52:45.208617 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:52:45.217133 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e1415-bd8d-4529-8302-916e430f2d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:52:45.218013 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e1415-bd8d-4529-8302-916e430f2d9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:52:45.218299 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e1415-bd8d-4529-8302-916e430f2d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:52:45.218576 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e1415-bd8d-4529-8302-916e430f2d9d admin admin] PATH_INFO: `/v3/projects/3ffa9da064c64346b5f227c28a89ea81` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:52:45.219847 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e1415-bd8d-4529-8302-916e430f2d9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ffa9da064c64346b5f227c28a89ea81)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:52:45.221767 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e1415-bd8d-4529-8302-916e430f2d9d admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:52:45.267907 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-316e1415-bd8d-4529-8302-916e430f2d9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ffa9da064c64346b5f227c28a89ea81', 'request_id': 'req-316e1415-bd8d-4529-8302-916e430f2d9d'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:52:45.274595 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-316e1415-bd8d-4529-8302-916e430f2d9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ffa9da064c64346b5f227c28a89ea81', 'request_id': 'req-316e1415-bd8d-4529-8302-916e430f2d9d'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:52:45.281811 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-316e1415-bd8d-4529-8302-916e430f2d9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ffa9da064c64346b5f227c28a89ea81', 'request_id': 'req-316e1415-bd8d-4529-8302-916e430f2d9d'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:52:45.292592 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-316e1415-bd8d-4529-8302-916e430f2d9d admin admin] The token cache is being invalidate because project 3ffa9da064c64346b5f227c28a89ea81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:52:45.292863 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-316e1415-bd8d-4529-8302-916e430f2d9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:52:45.293518 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-316e1415-bd8d-4529-8302-916e430f2d9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:52:45.295152 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1218/2448] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:52:45 2022] DELETE /identity/v3/projects/3ffa9da064c64346b5f227c28a89ea81 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:52:45.298999 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa9954-b000-4039-a62b-fee54e2737b6 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:52:45.300659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-03aa9954-b000-4039-a62b-fee54e2737b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:52:45.313733 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa9954-b000-4039-a62b-fee54e2737b6 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:52:45.368882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e685a2-f271-478a-91ad-5468bc34f687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:52:45.369777 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e685a2-f271-478a-91ad-5468bc34f687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:52:45.370065 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e685a2-f271-478a-91ad-5468bc34f687 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:52:45.370346 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e685a2-f271-478a-91ad-5468bc34f687 admin admin] PATH_INFO: `/v3/users/7dbe70ba377644ba9c146a092655b02e` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:52:45.371531 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e685a2-f271-478a-91ad-5468bc34f687 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dbe70ba377644ba9c146a092655b02e)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:52:45.373491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e685a2-f271-478a-91ad-5468bc34f687 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:52:45.419114 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-48e685a2-f271-478a-91ad-5468bc34f687 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dbe70ba377644ba9c146a092655b02e'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:52:45.426690 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-48e685a2-f271-478a-91ad-5468bc34f687 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dbe70ba377644ba9c146a092655b02e'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:52:45.433888 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-48e685a2-f271-478a-91ad-5468bc34f687 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dbe70ba377644ba9c146a092655b02e'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:52:45.434534 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-48e685a2-f271-478a-91ad-5468bc34f687 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dbe70ba377644ba9c146a092655b02e'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:52:45.446768 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-48e685a2-f271-478a-91ad-5468bc34f687 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dbe70ba377644ba9c146a092655b02e'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:52:45.452505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1231/2449] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:52:45 2022] DELETE /identity/v3/users/7dbe70ba377644ba9c146a092655b02e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:52:45.625840 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e1415-bd8d-4529-8302-916e430f2d9d admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:52:45.627364 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-316e1415-bd8d-4529-8302-916e430f2d9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:52:45.640742 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e1415-bd8d-4529-8302-916e430f2d9d admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:52:45.697799 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62ae8e-fab4-4414-8e5f-3d7378a912c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:52:45.698661 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62ae8e-fab4-4414-8e5f-3d7378a912c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:52:45.698942 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62ae8e-fab4-4414-8e5f-3d7378a912c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:52:45.699214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62ae8e-fab4-4414-8e5f-3d7378a912c4 admin admin] PATH_INFO: `/v3/projects/990f781d3ce943d2a69b9ca6ce3ea99c` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:52:45.700378 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd62ae8e-fab4-4414-8e5f-3d7378a912c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=990f781d3ce943d2a69b9ca6ce3ea99c)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:52:45.702396 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd62ae8e-fab4-4414-8e5f-3d7378a912c4 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:52:45.747134 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-bd62ae8e-fab4-4414-8e5f-3d7378a912c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '990f781d3ce943d2a69b9ca6ce3ea99c', 'request_id': 'req-bd62ae8e-fab4-4414-8e5f-3d7378a912c4'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:52:45.753081 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-bd62ae8e-fab4-4414-8e5f-3d7378a912c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '990f781d3ce943d2a69b9ca6ce3ea99c', 'request_id': 'req-bd62ae8e-fab4-4414-8e5f-3d7378a912c4'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:52:45.761182 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-bd62ae8e-fab4-4414-8e5f-3d7378a912c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '990f781d3ce943d2a69b9ca6ce3ea99c', 'request_id': 'req-bd62ae8e-fab4-4414-8e5f-3d7378a912c4'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:52:45.771168 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-bd62ae8e-fab4-4414-8e5f-3d7378a912c4 admin admin] The token cache is being invalidate because project 990f781d3ce943d2a69b9ca6ce3ea99c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:52:45.771452 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-bd62ae8e-fab4-4414-8e5f-3d7378a912c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:52:45.772060 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-bd62ae8e-fab4-4414-8e5f-3d7378a912c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:52:45.773701 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1219/2450] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:52:45 2022] DELETE /identity/v3/projects/990f781d3ce943d2a69b9ca6ce3ea99c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:52:55.449119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=61,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:52:55.768172 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=77,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:53:02.889692 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e685a2-f271-478a-91ad-5468bc34f687 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:53:02.891123 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-48e685a2-f271-478a-91ad-5468bc34f687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:53:02.947591 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e685a2-f271-478a-91ad-5468bc34f687 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:53:02.996225 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b0103-0403-4e04-b135-917c4c7f8fbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:53:02.996865 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b0103-0403-4e04-b135-917c4c7f8fbd service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:53:02.997183 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b0103-0403-4e04-b135-917c4c7f8fbd service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:53:02.997457 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b0103-0403-4e04-b135-917c4c7f8fbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:53:02.999215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-849b0103-0403-4e04-b135-917c4c7f8fbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:53:03.059426 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849b0103-0403-4e04-b135-917c4c7f8fbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:53:03.060843 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849b0103-0403-4e04-b135-917c4c7f8fbd service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:53:03.071766 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1232/2451] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:53:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:53:06.122822 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62ae8e-fab4-4414-8e5f-3d7378a912c4 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:53:06.124450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-bd62ae8e-fab4-4414-8e5f-3d7378a912c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:53:06.177522 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b0103-0403-4e04-b135-917c4c7f8fbd service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:53:06.178993 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-849b0103-0403-4e04-b135-917c4c7f8fbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:53:06.191521 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b0103-0403-4e04-b135-917c4c7f8fbd service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:53:06.196350 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b58b0-1b80-444d-9d40-c2b944b67092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:53:06.197259 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b58b0-1b80-444d-9d40-c2b944b67092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:53:06.197545 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b58b0-1b80-444d-9d40-c2b944b67092 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:53:06.197821 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b58b0-1b80-444d-9d40-c2b944b67092 admin admin] PATH_INFO: `/v3/users/0c83c05bec4f456d8aa7d51732ddc8af` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:53:06.199039 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501b58b0-1b80-444d-9d40-c2b944b67092 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c83c05bec4f456d8aa7d51732ddc8af)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:53:06.200948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501b58b0-1b80-444d-9d40-c2b944b67092 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:53:06.223543 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62ae8e-fab4-4414-8e5f-3d7378a912c4 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:53:06.230529 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4404ec-66cb-44b6-9080-c6bfcf8cbb77 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '890d2ae150d647a18dae25159cf97484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:53:06.231268 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4404ec-66cb-44b6-9080-c6bfcf8cbb77 service placement] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:53:06.231544 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4404ec-66cb-44b6-9080-c6bfcf8cbb77 service placement] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:53:06.231833 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4404ec-66cb-44b6-9080-c6bfcf8cbb77 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:53:06.233881 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-dc4404ec-66cb-44b6-9080-c6bfcf8cbb77 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:53:06.251503 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4404ec-66cb-44b6-9080-c6bfcf8cbb77 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:53:06.252554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-501b58b0-1b80-444d-9d40-c2b944b67092 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c83c05bec4f456d8aa7d51732ddc8af'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:53:06.253336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4404ec-66cb-44b6-9080-c6bfcf8cbb77 service placement] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:53:06.263694 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-501b58b0-1b80-444d-9d40-c2b944b67092 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c83c05bec4f456d8aa7d51732ddc8af'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:53:06.264869 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1220/2452] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:53:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:53:06.270239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:53:06.272587 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-501b58b0-1b80-444d-9d40-c2b944b67092 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c83c05bec4f456d8aa7d51732ddc8af'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:53:06.273502 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-501b58b0-1b80-444d-9d40-c2b944b67092 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c83c05bec4f456d8aa7d51732ddc8af'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:53:06.295342 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-501b58b0-1b80-444d-9d40-c2b944b67092 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c83c05bec4f456d8aa7d51732ddc8af'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:53:06.306223 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1233/2453] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:53:06 2022] DELETE /identity/v3/users/0c83c05bec4f456d8aa7d51732ddc8af => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:53:06.489586 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4404ec-66cb-44b6-9080-c6bfcf8cbb77 service placement] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:53:06.501049 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4404ec-66cb-44b6-9080-c6bfcf8cbb77 service placement] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:53:06.587560 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df03fd-0a07-48b9-814c-ac578a3ca2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:53:06.588326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df03fd-0a07-48b9-814c-ac578a3ca2d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:53:06.588610 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df03fd-0a07-48b9-814c-ac578a3ca2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:53:06.588886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df03fd-0a07-48b9-814c-ac578a3ca2d7 admin admin] PATH_INFO: `/v3/projects/3b2a80a004e04da7ae12db9f3f87146f` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:53:06.590134 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54df03fd-0a07-48b9-814c-ac578a3ca2d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b2a80a004e04da7ae12db9f3f87146f)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:53:06.592033 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54df03fd-0a07-48b9-814c-ac578a3ca2d7 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:53:06.639850 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-54df03fd-0a07-48b9-814c-ac578a3ca2d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b2a80a004e04da7ae12db9f3f87146f', 'request_id': 'req-54df03fd-0a07-48b9-814c-ac578a3ca2d7'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:53:06.646210 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-54df03fd-0a07-48b9-814c-ac578a3ca2d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b2a80a004e04da7ae12db9f3f87146f', 'request_id': 'req-54df03fd-0a07-48b9-814c-ac578a3ca2d7'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:53:06.654583 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-54df03fd-0a07-48b9-814c-ac578a3ca2d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b2a80a004e04da7ae12db9f3f87146f', 'request_id': 'req-54df03fd-0a07-48b9-814c-ac578a3ca2d7'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:53:06.665360 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-54df03fd-0a07-48b9-814c-ac578a3ca2d7 admin admin] The token cache is being invalidate because project 3b2a80a004e04da7ae12db9f3f87146f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:53:06.665653 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-54df03fd-0a07-48b9-814c-ac578a3ca2d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:53:06.666897 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-54df03fd-0a07-48b9-814c-ac578a3ca2d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:53:06.668484 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1221/2454] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:53:06 2022] DELETE /identity/v3/projects/3b2a80a004e04da7ae12db9f3f87146f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:53:06.673470 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643e554-c260-4d0a-891e-b0b4c9c0e127 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:53:06.673767 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643e554-c260-4d0a-891e-b0b4c9c0e127 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:53:06.674303 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643e554-c260-4d0a-891e-b0b4c9c0e127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:53:06.692773 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-2643e554-c260-4d0a-891e-b0b4c9c0e127 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:53:06.694124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-2643e554-c260-4d0a-891e-b0b4c9c0e127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:53:06.755420 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1234/2455] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:53:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:53:06.764013 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f6ccc-42c0-4c5b-b4c7-7d6f0d03c6c9 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:53:06.764303 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f6ccc-42c0-4c5b-b4c7-7d6f0d03c6c9 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:53:06.764584 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f6ccc-42c0-4c5b-b4c7-7d6f0d03c6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:53:06.783115 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-408f6ccc-42c0-4c5b-b4c7-7d6f0d03c6c9 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:53:06.784305 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-408f6ccc-42c0-4c5b-b4c7-7d6f0d03c6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:53:06.792349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1222/2456] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:53:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:53:06.796562 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2643e554-c260-4d0a-891e-b0b4c9c0e127 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:53:06.806700 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2643e554-c260-4d0a-891e-b0b4c9c0e127 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:53:06.811241 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62982b29-9579-4168-bc8d-0501ec4cb326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:53:06.811891 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62982b29-9579-4168-bc8d-0501ec4cb326 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:53:06.812173 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62982b29-9579-4168-bc8d-0501ec4cb326 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:53:06.812452 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62982b29-9579-4168-bc8d-0501ec4cb326 admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:53:06.813209 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62982b29-9579-4168-bc8d-0501ec4cb326 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:53:06.813521 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62982b29-9579-4168-bc8d-0501ec4cb326 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:53:06.815103 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62982b29-9579-4168-bc8d-0501ec4cb326 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:53:06.827041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1235/2457] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:53:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:53:06.831160 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f6ccc-42c0-4c5b-b4c7-7d6f0d03c6c9 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:53:06.840877 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f6ccc-42c0-4c5b-b4c7-7d6f0d03c6c9 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:53:06.845619 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4cde3-c75f-4249-94d1-47a45a7e6be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:53:06.846270 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4cde3-c75f-4249-94d1-47a45a7e6be3 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:53:06.846554 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4cde3-c75f-4249-94d1-47a45a7e6be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:53:06.846828 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4cde3-c75f-4249-94d1-47a45a7e6be3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:53:06.847577 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d4cde3-c75f-4249-94d1-47a45a7e6be3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:53:06.849551 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d4cde3-c75f-4249-94d1-47a45a7e6be3 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:53:06.859661 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:53:06.893025 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1223/2458] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:53:06 2022] POST /identity/v3/projects => generated 387 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:53:06.898098 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62982b29-9579-4168-bc8d-0501ec4cb326 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:53:06.908453 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62982b29-9579-4168-bc8d-0501ec4cb326 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:53:06.966056 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19a565-2af2-4b91-aca1-a5b2203e91e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:53:06.966719 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19a565-2af2-4b91-aca1-a5b2203e91e6 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:53:06.967005 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19a565-2af2-4b91-aca1-a5b2203e91e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:53:06.967288 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19a565-2af2-4b91-aca1-a5b2203e91e6 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:53:06.967989 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe19a565-2af2-4b91-aca1-a5b2203e91e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:53:06.969929 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe19a565-2af2-4b91-aca1-a5b2203e91e6 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:53:06.993373 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1236/2459] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:53:06 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:53:06.994539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4cde3-c75f-4249-94d1-47a45a7e6be3 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:53:07.004760 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4cde3-c75f-4249-94d1-47a45a7e6be3 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:53:07.009926 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a1761-69de-4d66-885a-f456811f5e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:53:07.010578 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a1761-69de-4d66-885a-f456811f5e0c admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:53:07.010859 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a1761-69de-4d66-885a-f456811f5e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:53:07.011128 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a1761-69de-4d66-885a-f456811f5e0c admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:53:07.011843 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546a1761-69de-4d66-885a-f456811f5e0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:53:07.013483 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546a1761-69de-4d66-885a-f456811f5e0c admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:53:07.023221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1224/2460] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:53:07.027085 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19a565-2af2-4b91-aca1-a5b2203e91e6 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:53:07.036910 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19a565-2af2-4b91-aca1-a5b2203e91e6 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:53:07.041516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c31ae30-a702-43e0-ac05-2b5af6f9e3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:53:07.042165 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31ae30-a702-43e0-ac05-2b5af6f9e3d4 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:53:07.042446 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31ae30-a702-43e0-ac05-2b5af6f9e3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:53:07.042719 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31ae30-a702-43e0-ac05-2b5af6f9e3d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:53:07.043440 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c31ae30-a702-43e0-ac05-2b5af6f9e3d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:53:07.045086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c31ae30-a702-43e0-ac05-2b5af6f9e3d4 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:53:07.055599 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1237/2461] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:53:07.059718 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a1761-69de-4d66-885a-f456811f5e0c admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:53:07.069968 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a1761-69de-4d66-885a-f456811f5e0c admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:53:07.075156 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a2831-06b8-4272-840b-3abfd2a62d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:53:07.076028 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a2831-06b8-4272-840b-3abfd2a62d7a admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:53:07.076718 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a2831-06b8-4272-840b-3abfd2a62d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:53:07.076718 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a2831-06b8-4272-840b-3abfd2a62d7a admin admin] PATH_INFO: `/v3/projects/9e3edb8d53184b98b7055ffc145f186d/users/c85251638bc64deca16008b696b11ea4/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:53:07.091777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289a2831-06b8-4272-840b-3abfd2a62d7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e3edb8d53184b98b7055ffc145f186d, user_id=c85251638bc64deca16008b696b11ea4, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:53:07.094458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289a2831-06b8-4272-840b-3abfd2a62d7a admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:53:07.108754 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1225/2462] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:53:07 2022] PUT /identity/v3/projects/9e3edb8d53184b98b7055ffc145f186d/users/c85251638bc64deca16008b696b11ea4/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:53:07.114217 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639881a-9f5f-49bc-8b0d-678622a75391 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:53:07.114518 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639881a-9f5f-49bc-8b0d-678622a75391 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:53:07.114806 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639881a-9f5f-49bc-8b0d-678622a75391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:53:07.134635 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-9639881a-9f5f-49bc-8b0d-678622a75391 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:53:07.135852 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-9639881a-9f5f-49bc-8b0d-678622a75391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:53:07.176770 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:53:07.200173 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1238/2463] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:53:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:53:07.211014 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a2831-06b8-4272-840b-3abfd2a62d7a admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:53:07.212862 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-289a2831-06b8-4272-840b-3abfd2a62d7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:53:07.273893 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a2831-06b8-4272-840b-3abfd2a62d7a admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:53:07.277479 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8a0a5-bdfc-496e-ac85-377db5aa7e6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:53:07.278121 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8a0a5-bdfc-496e-ac85-377db5aa7e6d service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:53:07.278403 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8a0a5-bdfc-496e-ac85-377db5aa7e6d service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:53:07.278677 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8a0a5-bdfc-496e-ac85-377db5aa7e6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:53:07.287438 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d8a0a5-bdfc-496e-ac85-377db5aa7e6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:53:07.289547 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d8a0a5-bdfc-496e-ac85-377db5aa7e6d service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:53:07.299643 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1226/2464] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:53:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:53:11.876875 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923b1c7-08cf-4153-9409-dda2f94f46d6 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:53:11.877412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923b1c7-08cf-4153-9409-dda2f94f46d6 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:53:11.877542 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923b1c7-08cf-4153-9409-dda2f94f46d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:53:11.911215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-9923b1c7-08cf-4153-9409-dda2f94f46d6 None None] MFA Rules not processed for user `c85251638bc64deca16008b696b11ea4`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:53:11.912468 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-9923b1c7-08cf-4153-9409-dda2f94f46d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:53:12.006468 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1239/2465] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:53:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:53:12.021231 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8a0a5-bdfc-496e-ac85-377db5aa7e6d service neutron] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:53:12.022711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-21d8a0a5-bdfc-496e-ac85-377db5aa7e6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:53:12.129062 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8a0a5-bdfc-496e-ac85-377db5aa7e6d service neutron] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:53:12.134925 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d31efd-3371-40db-b265-795375a9fe77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:53:12.135924 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d31efd-3371-40db-b265-795375a9fe77 service nova] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:53:12.136274 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d31efd-3371-40db-b265-795375a9fe77 service nova] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:53:12.136715 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d31efd-3371-40db-b265-795375a9fe77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:53:12.145911 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d31efd-3371-40db-b265-795375a9fe77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:53:12.147527 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d31efd-3371-40db-b265-795375a9fe77 service nova] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:53:12.158259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1227/2466] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:53:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:53:12.753214 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90038d1-97fd-402a-a81a-4f65022212ae None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:53:12.753509 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90038d1-97fd-402a-a81a-4f65022212ae None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:53:12.753793 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90038d1-97fd-402a-a81a-4f65022212ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:53:12.771808 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-a90038d1-97fd-402a-a81a-4f65022212ae None None] MFA Rules not processed for user `3d7cacb73c8342579d9f990b0193487c`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:53:12.773930 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-a90038d1-97fd-402a-a81a-4f65022212ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:53:12.832315 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1240/2467] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:53:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:53:13.863336 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257711f7-68ec-4707-ba58-5f2318fee6b9 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:53:13.864363 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257711f7-68ec-4707-ba58-5f2318fee6b9 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:53:13.864864 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257711f7-68ec-4707-ba58-5f2318fee6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:53:13.894183 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-257711f7-68ec-4707-ba58-5f2318fee6b9 None None] MFA Rules not processed for user `c85251638bc64deca16008b696b11ea4`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:53:13.897365 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-257711f7-68ec-4707-ba58-5f2318fee6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:53:13.937119 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1228/2468] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:53:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:53:19.749750 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90038d1-97fd-402a-a81a-4f65022212ae None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:53:19.751219 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-a90038d1-97fd-402a-a81a-4f65022212ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:53:19.764037 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90038d1-97fd-402a-a81a-4f65022212ae None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:53:19.767598 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4959eb0-9f47-4338-822c-8b3ec7c95d06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:53:19.768242 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4959eb0-9f47-4338-822c-8b3ec7c95d06 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:53:19.768519 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4959eb0-9f47-4338-822c-8b3ec7c95d06 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:53:19.768794 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4959eb0-9f47-4338-822c-8b3ec7c95d06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:53:19.770567 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-e4959eb0-9f47-4338-822c-8b3ec7c95d06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:53:19.783106 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4959eb0-9f47-4338-822c-8b3ec7c95d06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:53:19.784556 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4959eb0-9f47-4338-822c-8b3ec7c95d06 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:53:19.794752 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1241/2469] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:53:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:53:23.880031 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=85,keystone:INSERT=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:53:27.820935 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257711f7-68ec-4707-ba58-5f2318fee6b9 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:53:27.822512 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-257711f7-68ec-4707-ba58-5f2318fee6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:53:27.842178 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257711f7-68ec-4707-ba58-5f2318fee6b9 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:53:27.848629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c82519-87dc-4be8-8d88-7f2947a71fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:53:27.849629 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c82519-87dc-4be8-8d88-7f2947a71fa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:53:27.849978 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c82519-87dc-4be8-8d88-7f2947a71fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:53:27.850938 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c82519-87dc-4be8-8d88-7f2947a71fa3 admin admin] PATH_INFO: `/v3/users/53d00e4198e240c8a5c2fbe8bd241b45` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:53:27.852375 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c82519-87dc-4be8-8d88-7f2947a71fa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53d00e4198e240c8a5c2fbe8bd241b45)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:53:27.854406 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c82519-87dc-4be8-8d88-7f2947a71fa3 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:53:27.914191 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a9c82519-87dc-4be8-8d88-7f2947a71fa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53d00e4198e240c8a5c2fbe8bd241b45'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:53:27.922010 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a9c82519-87dc-4be8-8d88-7f2947a71fa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53d00e4198e240c8a5c2fbe8bd241b45'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:53:27.922637 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a9c82519-87dc-4be8-8d88-7f2947a71fa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53d00e4198e240c8a5c2fbe8bd241b45'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:53:27.930399 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a9c82519-87dc-4be8-8d88-7f2947a71fa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53d00e4198e240c8a5c2fbe8bd241b45'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:53:27.942416 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a9c82519-87dc-4be8-8d88-7f2947a71fa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53d00e4198e240c8a5c2fbe8bd241b45'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:53:27.948731 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1229/2470] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:53:27 2022] DELETE /identity/v3/users/53d00e4198e240c8a5c2fbe8bd241b45 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:53:28.123813 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4959eb0-9f47-4338-822c-8b3ec7c95d06 service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:53:28.135278 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4959eb0-9f47-4338-822c-8b3ec7c95d06 service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:53:28.198324 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc112c-bfad-47ba-b6be-68295d4dd914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:53:28.199213 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc112c-bfad-47ba-b6be-68295d4dd914 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:53:28.199497 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc112c-bfad-47ba-b6be-68295d4dd914 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:53:28.199778 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc112c-bfad-47ba-b6be-68295d4dd914 admin admin] PATH_INFO: `/v3/projects/1727f66e37f94cc6971b574c8635e078` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:53:28.200984 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc112c-bfad-47ba-b6be-68295d4dd914 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1727f66e37f94cc6971b574c8635e078)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:53:28.203307 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc112c-bfad-47ba-b6be-68295d4dd914 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:53:28.212395 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:53:28.252415 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-2dfc112c-bfad-47ba-b6be-68295d4dd914 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1727f66e37f94cc6971b574c8635e078', 'request_id': 'req-2dfc112c-bfad-47ba-b6be-68295d4dd914'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:53:28.259030 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-2dfc112c-bfad-47ba-b6be-68295d4dd914 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1727f66e37f94cc6971b574c8635e078', 'request_id': 'req-2dfc112c-bfad-47ba-b6be-68295d4dd914'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:53:28.266472 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-2dfc112c-bfad-47ba-b6be-68295d4dd914 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1727f66e37f94cc6971b574c8635e078', 'request_id': 'req-2dfc112c-bfad-47ba-b6be-68295d4dd914'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:53:28.277990 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-2dfc112c-bfad-47ba-b6be-68295d4dd914 admin admin] The token cache is being invalidate because project 1727f66e37f94cc6971b574c8635e078 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:53:28.278274 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-2dfc112c-bfad-47ba-b6be-68295d4dd914 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:53:28.278887 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-2dfc112c-bfad-47ba-b6be-68295d4dd914 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:53:28.280415 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1242/2471] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:53:28 2022] DELETE /identity/v3/projects/1727f66e37f94cc6971b574c8635e078 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:53:28.284279 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c82519-87dc-4be8-8d88-7f2947a71fa3 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:53:28.285664 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a9c82519-87dc-4be8-8d88-7f2947a71fa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:53:28.299249 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c82519-87dc-4be8-8d88-7f2947a71fa3 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:53:28.357572 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccdaa48-ee84-46c6-bd66-c45a00a3bd52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:53:28.358359 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccdaa48-ee84-46c6-bd66-c45a00a3bd52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:53:28.358641 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccdaa48-ee84-46c6-bd66-c45a00a3bd52 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:53:28.358942 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccdaa48-ee84-46c6-bd66-c45a00a3bd52 admin admin] PATH_INFO: `/v3/users/1abf4a9a40b94ed88ac15703cd5dc36f` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:53:28.360115 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccdaa48-ee84-46c6-bd66-c45a00a3bd52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1abf4a9a40b94ed88ac15703cd5dc36f)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:53:28.362069 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccdaa48-ee84-46c6-bd66-c45a00a3bd52 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:53:28.407764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-0ccdaa48-ee84-46c6-bd66-c45a00a3bd52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1abf4a9a40b94ed88ac15703cd5dc36f'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:53:28.414874 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-0ccdaa48-ee84-46c6-bd66-c45a00a3bd52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1abf4a9a40b94ed88ac15703cd5dc36f'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:53:28.415492 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-0ccdaa48-ee84-46c6-bd66-c45a00a3bd52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1abf4a9a40b94ed88ac15703cd5dc36f'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:53:28.422444 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-0ccdaa48-ee84-46c6-bd66-c45a00a3bd52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1abf4a9a40b94ed88ac15703cd5dc36f'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:53:28.430325 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:53:28.435184 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-0ccdaa48-ee84-46c6-bd66-c45a00a3bd52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1abf4a9a40b94ed88ac15703cd5dc36f'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:53:28.442230 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1230/2472] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:53:28 2022] DELETE /identity/v3/users/1abf4a9a40b94ed88ac15703cd5dc36f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:53:28.618659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc112c-bfad-47ba-b6be-68295d4dd914 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:53:28.620408 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-2dfc112c-bfad-47ba-b6be-68295d4dd914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:53:28.634899 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc112c-bfad-47ba-b6be-68295d4dd914 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:53:28.704079 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044bfef-c048-4847-b300-6cf6ecc24436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:53:28.705065 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044bfef-c048-4847-b300-6cf6ecc24436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:53:28.705425 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044bfef-c048-4847-b300-6cf6ecc24436 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:53:28.705775 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044bfef-c048-4847-b300-6cf6ecc24436 admin admin] PATH_INFO: `/v3/projects/0a185cea242c43d3a0919fc48a32f681` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:53:28.707092 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c044bfef-c048-4847-b300-6cf6ecc24436 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a185cea242c43d3a0919fc48a32f681)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:53:28.709108 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c044bfef-c048-4847-b300-6cf6ecc24436 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:53:28.768820 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c044bfef-c048-4847-b300-6cf6ecc24436 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a185cea242c43d3a0919fc48a32f681', 'request_id': 'req-c044bfef-c048-4847-b300-6cf6ecc24436'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:53:28.775049 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c044bfef-c048-4847-b300-6cf6ecc24436 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a185cea242c43d3a0919fc48a32f681', 'request_id': 'req-c044bfef-c048-4847-b300-6cf6ecc24436'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:53:28.783278 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c044bfef-c048-4847-b300-6cf6ecc24436 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a185cea242c43d3a0919fc48a32f681', 'request_id': 'req-c044bfef-c048-4847-b300-6cf6ecc24436'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:53:28.795826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c044bfef-c048-4847-b300-6cf6ecc24436 admin admin] The token cache is being invalidate because project 0a185cea242c43d3a0919fc48a32f681 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:53:28.796118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c044bfef-c048-4847-b300-6cf6ecc24436 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:53:28.796756 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c044bfef-c048-4847-b300-6cf6ecc24436 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:53:28.798320 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1243/2473] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:53:28 2022] DELETE /identity/v3/projects/0a185cea242c43d3a0919fc48a32f681 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:53:38.441341 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:53:38.792745 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=72,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:53:39.660388 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f6eda2-c4d8-4cd4-b564-a456bb4a9764 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:53:39.660861 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f6eda2-c4d8-4cd4-b564-a456bb4a9764 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:53:39.660957 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f6eda2-c4d8-4cd4-b564-a456bb4a9764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:53:39.679568 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-47f6eda2-c4d8-4cd4-b564-a456bb4a9764 None None] MFA Rules not processed for user `7b572a8f465048558fc969b85b2aec5b`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:53:39.680745 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-47f6eda2-c4d8-4cd4-b564-a456bb4a9764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:53:39.734099 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1231/2474] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:53:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:53:40.034235 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044bfef-c048-4847-b300-6cf6ecc24436 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:53:40.035647 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-c044bfef-c048-4847-b300-6cf6ecc24436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:53:40.093124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044bfef-c048-4847-b300-6cf6ecc24436 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:53:40.096731 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83857f9-23c5-4f56-9c20-c9981282a6d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:53:40.097400 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83857f9-23c5-4f56-9c20-c9981282a6d0 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:53:40.097684 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83857f9-23c5-4f56-9c20-c9981282a6d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:53:40.097961 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83857f9-23c5-4f56-9c20-c9981282a6d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:53:40.099720 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-d83857f9-23c5-4f56-9c20-c9981282a6d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:53:40.109930 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83857f9-23c5-4f56-9c20-c9981282a6d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:53:40.111334 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83857f9-23c5-4f56-9c20-c9981282a6d0 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:53:40.119249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1244/2475] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:53:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:53:49.727305 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=25 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:53:49.763755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0289899-90a0-4afb-b1a3-99ead80c0900 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:53:49.764260 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0289899-90a0-4afb-b1a3-99ead80c0900 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:53:49.764765 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0289899-90a0-4afb-b1a3-99ead80c0900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:53:49.786736 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-a0289899-90a0-4afb-b1a3-99ead80c0900 None None] MFA Rules not processed for user `32a8191b1c184c75b428579ace2da61c`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:53:49.788692 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a0289899-90a0-4afb-b1a3-99ead80c0900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:53:49.846195 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1232/2476] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:53:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:53:50.115778 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=26 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:53:59.839450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=25 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:54:26.985235 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83857f9-23c5-4f56-9c20-c9981282a6d0 service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:54:26.986625 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-d83857f9-23c5-4f56-9c20-c9981282a6d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:54:27.054765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83857f9-23c5-4f56-9c20-c9981282a6d0 service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:54:27.059654 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee261bb-f883-4af3-a82c-e44b1b8c7c78 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '890d2ae150d647a18dae25159cf97484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:54:27.060287 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee261bb-f883-4af3-a82c-e44b1b8c7c78 service placement] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:54:27.060572 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee261bb-f883-4af3-a82c-e44b1b8c7c78 service placement] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:54:27.060848 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee261bb-f883-4af3-a82c-e44b1b8c7c78 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:54:27.062594 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-6ee261bb-f883-4af3-a82c-e44b1b8c7c78 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:54:27.074469 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee261bb-f883-4af3-a82c-e44b1b8c7c78 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:54:27.076023 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee261bb-f883-4af3-a82c-e44b1b8c7c78 service placement] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:54:27.085761 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1245/2477] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:54:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:54:37.081517 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=32 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:55:23.432200 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96477a-6a06-4297-b1f6-49940fcb4f0d None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:23.432776 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96477a-6a06-4297-b1f6-49940fcb4f0d None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:23.432977 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96477a-6a06-4297-b1f6-49940fcb4f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:23.455428 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-7a96477a-6a06-4297-b1f6-49940fcb4f0d None None] MFA Rules not processed for user `c4535134181e489fb532f617264d5855`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:55:23.456537 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-7a96477a-6a06-4297-b1f6-49940fcb4f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:23.574432 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1233/2478] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Jul 6 13:55:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:55:23.657958 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee261bb-f883-4af3-a82c-e44b1b8c7c78 service placement] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:23.660216 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-6ee261bb-f883-4af3-a82c-e44b1b8c7c78 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:23.741666 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee261bb-f883-4af3-a82c-e44b1b8c7c78 service placement] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:23.746971 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f0e116-4b9a-4cfb-8869-7fa5f1c579d5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c1bea05c37646b0a05fa8dda0367ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:23.748757 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0e116-4b9a-4cfb-8869-7fa5f1c579d5 service swift] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:23.749072 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0e116-4b9a-4cfb-8869-7fa5f1c579d5 service swift] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:23.749353 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0e116-4b9a-4cfb-8869-7fa5f1c579d5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:23.769722 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f0e116-4b9a-4cfb-8869-7fa5f1c579d5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:23.771585 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f0e116-4b9a-4cfb-8869-7fa5f1c579d5 service swift] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:23.780505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1246/2479] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jul 6 13:55:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 13:55:30.496292 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96477a-6a06-4297-b1f6-49940fcb4f0d None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:30.507961 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96477a-6a06-4297-b1f6-49940fcb4f0d None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:30.513634 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19371aa-ec5d-499e-8826-b835c480fcfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:30.514427 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19371aa-ec5d-499e-8826-b835c480fcfb service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:30.514931 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19371aa-ec5d-499e-8826-b835c480fcfb service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:30.515302 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19371aa-ec5d-499e-8826-b835c480fcfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:30.517257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a19371aa-ec5d-499e-8826-b835c480fcfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:30.593494 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19371aa-ec5d-499e-8826-b835c480fcfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:30.595098 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19371aa-ec5d-499e-8826-b835c480fcfb service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:30.607757 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1234/2480] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:55:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:55:33.779192 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=32 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:55:34.710233 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f0e116-4b9a-4cfb-8869-7fa5f1c579d5 service swift] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:34.711707 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-48f0e116-4b9a-4cfb-8869-7fa5f1c579d5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:34.726482 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f0e116-4b9a-4cfb-8869-7fa5f1c579d5 service swift] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:34.730950 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88c02c-9400-4a6e-8e27-2007f94355a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:34.732605 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88c02c-9400-4a6e-8e27-2007f94355a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:34.732991 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88c02c-9400-4a6e-8e27-2007f94355a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:34.733298 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88c02c-9400-4a6e-8e27-2007f94355a5 admin admin] PATH_INFO: `/v3/users/3d7cacb73c8342579d9f990b0193487c` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:34.747951 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88c02c-9400-4a6e-8e27-2007f94355a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d7cacb73c8342579d9f990b0193487c)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:34.749910 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88c02c-9400-4a6e-8e27-2007f94355a5 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:34.799765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-9d88c02c-9400-4a6e-8e27-2007f94355a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d7cacb73c8342579d9f990b0193487c'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:34.807653 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-9d88c02c-9400-4a6e-8e27-2007f94355a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d7cacb73c8342579d9f990b0193487c'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:34.814921 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-9d88c02c-9400-4a6e-8e27-2007f94355a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d7cacb73c8342579d9f990b0193487c'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:34.815723 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-9d88c02c-9400-4a6e-8e27-2007f94355a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d7cacb73c8342579d9f990b0193487c'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:34.829122 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-9d88c02c-9400-4a6e-8e27-2007f94355a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d7cacb73c8342579d9f990b0193487c'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:34.835230 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1247/2481] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:55:34 2022] DELETE /identity/v3/users/3d7cacb73c8342579d9f990b0193487c => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:55:35.052002 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19371aa-ec5d-499e-8826-b835c480fcfb service neutron] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:35.067870 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19371aa-ec5d-499e-8826-b835c480fcfb service neutron] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:35.129842 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff117e94-6a3a-4a2f-9e07-130f6d0cfde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:35.130620 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff117e94-6a3a-4a2f-9e07-130f6d0cfde5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:35.130900 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff117e94-6a3a-4a2f-9e07-130f6d0cfde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:35.131171 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff117e94-6a3a-4a2f-9e07-130f6d0cfde5 admin admin] PATH_INFO: `/v3/projects/05df572327d54bd99595974b75898052` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:35.141596 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff117e94-6a3a-4a2f-9e07-130f6d0cfde5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05df572327d54bd99595974b75898052)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:35.143498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff117e94-6a3a-4a2f-9e07-130f6d0cfde5 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:35.181065 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:55:35.195490 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-ff117e94-6a3a-4a2f-9e07-130f6d0cfde5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05df572327d54bd99595974b75898052', 'request_id': 'req-ff117e94-6a3a-4a2f-9e07-130f6d0cfde5'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:35.202576 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-ff117e94-6a3a-4a2f-9e07-130f6d0cfde5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05df572327d54bd99595974b75898052', 'request_id': 'req-ff117e94-6a3a-4a2f-9e07-130f6d0cfde5'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:35.210160 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-ff117e94-6a3a-4a2f-9e07-130f6d0cfde5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05df572327d54bd99595974b75898052', 'request_id': 'req-ff117e94-6a3a-4a2f-9e07-130f6d0cfde5'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:35.222534 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-ff117e94-6a3a-4a2f-9e07-130f6d0cfde5 admin admin] The token cache is being invalidate because project 05df572327d54bd99595974b75898052 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:55:35.222882 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-ff117e94-6a3a-4a2f-9e07-130f6d0cfde5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:35.223543 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-ff117e94-6a3a-4a2f-9e07-130f6d0cfde5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:35.225131 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1235/2482] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:55:35 2022] DELETE /identity/v3/projects/05df572327d54bd99595974b75898052 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:55:35.230049 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b9ae2-4f8e-48b7-9887-b073ac734942 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:35.230340 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b9ae2-4f8e-48b7-9887-b073ac734942 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:35.230681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b9ae2-4f8e-48b7-9887-b073ac734942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:35.249300 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-b58b9ae2-4f8e-48b7-9887-b073ac734942 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:55:35.250584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-b58b9ae2-4f8e-48b7-9887-b073ac734942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:35.317541 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1248/2483] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:55:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:55:35.325804 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e13ac-9b6d-4ba7-8333-e05ec221b3f6 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:35.325804 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e13ac-9b6d-4ba7-8333-e05ec221b3f6 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:35.326581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e13ac-9b6d-4ba7-8333-e05ec221b3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:35.344699 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-0b0e13ac-9b6d-4ba7-8333-e05ec221b3f6 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:55:35.345928 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-0b0e13ac-9b6d-4ba7-8333-e05ec221b3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:35.354785 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1236/2484] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:55:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:55:35.359834 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b9ae2-4f8e-48b7-9887-b073ac734942 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:35.369630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b9ae2-4f8e-48b7-9887-b073ac734942 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:35.374560 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564fe75-b3dd-4db0-b6be-0923bc18b0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:35.375222 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564fe75-b3dd-4db0-b6be-0923bc18b0eb admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:35.375504 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564fe75-b3dd-4db0-b6be-0923bc18b0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:35.375778 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564fe75-b3dd-4db0-b6be-0923bc18b0eb admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:35.376510 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564fe75-b3dd-4db0-b6be-0923bc18b0eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:55:35.376813 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564fe75-b3dd-4db0-b6be-0923bc18b0eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:35.378628 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564fe75-b3dd-4db0-b6be-0923bc18b0eb admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:35.390608 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1249/2485] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:55:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:55:35.394930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e13ac-9b6d-4ba7-8333-e05ec221b3f6 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:35.404615 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e13ac-9b6d-4ba7-8333-e05ec221b3f6 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:35.409247 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8fcb38-f474-4357-8360-4dff87d83f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:35.409904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fcb38-f474-4357-8360-4dff87d83f63 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:35.410185 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fcb38-f474-4357-8360-4dff87d83f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:35.410457 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fcb38-f474-4357-8360-4dff87d83f63 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:35.411208 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8fcb38-f474-4357-8360-4dff87d83f63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:35.413120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8fcb38-f474-4357-8360-4dff87d83f63 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:35.441143 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1237/2486] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:55:35 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:55:35.443808 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564fe75-b3dd-4db0-b6be-0923bc18b0eb admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:35.453941 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564fe75-b3dd-4db0-b6be-0923bc18b0eb admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:35.499773 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:55:35.513286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5128f-25da-4878-904b-aa89af6e7ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:35.514219 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5128f-25da-4878-904b-aa89af6e7ee1 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:35.514590 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5128f-25da-4878-904b-aa89af6e7ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:35.514939 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5128f-25da-4878-904b-aa89af6e7ee1 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:35.515647 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e5128f-25da-4878-904b-aa89af6e7ee1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:35.517592 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e5128f-25da-4878-904b-aa89af6e7ee1 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:35.542226 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1250/2487] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:55:35 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:55:35.543517 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8fcb38-f474-4357-8360-4dff87d83f63 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:35.555055 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8fcb38-f474-4357-8360-4dff87d83f63 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:35.560054 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a9968-0fd2-4cf9-976c-692e4008848b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:35.560720 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a9968-0fd2-4cf9-976c-692e4008848b admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:35.561324 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a9968-0fd2-4cf9-976c-692e4008848b admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:35.561324 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a9968-0fd2-4cf9-976c-692e4008848b admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:35.562033 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0a9968-0fd2-4cf9-976c-692e4008848b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:35.563627 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0a9968-0fd2-4cf9-976c-692e4008848b admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:35.575892 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1238/2488] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:55:35.582148 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5128f-25da-4878-904b-aa89af6e7ee1 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:35.593408 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5128f-25da-4878-904b-aa89af6e7ee1 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:35.598310 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4318d5-3d9a-49cf-a705-2685cb7d2dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:35.599032 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4318d5-3d9a-49cf-a705-2685cb7d2dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:35.599321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4318d5-3d9a-49cf-a705-2685cb7d2dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:35.599599 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4318d5-3d9a-49cf-a705-2685cb7d2dc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:35.600336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4318d5-3d9a-49cf-a705-2685cb7d2dc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:35.602019 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4318d5-3d9a-49cf-a705-2685cb7d2dc9 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:35.617539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a9968-0fd2-4cf9-976c-692e4008848b admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:35.621261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1251/2489] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:55:35.629391 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a9968-0fd2-4cf9-976c-692e4008848b admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:35.634401 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3a1b5-e4eb-4dc3-b59e-e18616ed3a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:35.638421 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3a1b5-e4eb-4dc3-b59e-e18616ed3a1a admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:35.638421 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3a1b5-e4eb-4dc3-b59e-e18616ed3a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:35.638421 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3a1b5-e4eb-4dc3-b59e-e18616ed3a1a admin admin] PATH_INFO: `/v3/projects/13d9ba29cdb34d11bf82fa510e7a7a29/users/5e232b13cfe84689ae7ee06aacb7f813/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:35.656775 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c3a1b5-e4eb-4dc3-b59e-e18616ed3a1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=13d9ba29cdb34d11bf82fa510e7a7a29, user_id=5e232b13cfe84689ae7ee06aacb7f813, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:35.659980 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c3a1b5-e4eb-4dc3-b59e-e18616ed3a1a admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:35.686311 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1239/2490] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:55:35 2022] PUT /identity/v3/projects/13d9ba29cdb34d11bf82fa510e7a7a29/users/5e232b13cfe84689ae7ee06aacb7f813/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:55:35.688076 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4fa944-c68d-4513-bf5c-0dd52a5a40d0 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:35.688367 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4fa944-c68d-4513-bf5c-0dd52a5a40d0 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:35.688642 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4fa944-c68d-4513-bf5c-0dd52a5a40d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:35.723761 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-2e4fa944-c68d-4513-bf5c-0dd52a5a40d0 None None] MFA Rules not processed for user `5e232b13cfe84689ae7ee06aacb7f813`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:55:35.725176 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-2e4fa944-c68d-4513-bf5c-0dd52a5a40d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:35.795792 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1252/2491] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:55:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:55:35.800652 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3a1b5-e4eb-4dc3-b59e-e18616ed3a1a admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:35.862322 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3a1b5-e4eb-4dc3-b59e-e18616ed3a1a admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:35.866939 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea16377-e7e0-4856-a58a-3d74e8f9573e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:35.867592 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea16377-e7e0-4856-a58a-3d74e8f9573e admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:35.867875 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea16377-e7e0-4856-a58a-3d74e8f9573e admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:35.868149 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea16377-e7e0-4856-a58a-3d74e8f9573e admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:35.868845 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea16377-e7e0-4856-a58a-3d74e8f9573e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:35.870789 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea16377-e7e0-4856-a58a-3d74e8f9573e admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:35.897683 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1240/2492] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:55:35 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:55:35.901875 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4fa944-c68d-4513-bf5c-0dd52a5a40d0 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:35.914236 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4fa944-c68d-4513-bf5c-0dd52a5a40d0 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:35.979012 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4a750-73d4-4b46-a007-2846983da704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:35.979852 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4a750-73d4-4b46-a007-2846983da704 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:35.980270 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4a750-73d4-4b46-a007-2846983da704 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:35.980567 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4a750-73d4-4b46-a007-2846983da704 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:35.981380 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd4a750-73d4-4b46-a007-2846983da704 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:35.983751 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd4a750-73d4-4b46-a007-2846983da704 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:36.010494 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1253/2493] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:55:35 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:55:36.012262 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea16377-e7e0-4856-a58a-3d74e8f9573e admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:36.023918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea16377-e7e0-4856-a58a-3d74e8f9573e admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:36.028885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca5dbb-d041-42e0-b538-e9ec158cd5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:36.029693 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca5dbb-d041-42e0-b538-e9ec158cd5c9 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:36.029986 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca5dbb-d041-42e0-b538-e9ec158cd5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:36.030264 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca5dbb-d041-42e0-b538-e9ec158cd5c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:36.031005 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca5dbb-d041-42e0-b538-e9ec158cd5c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:36.032914 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca5dbb-d041-42e0-b538-e9ec158cd5c9 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:36.045649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1241/2494] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:55:36.049953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4a750-73d4-4b46-a007-2846983da704 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:36.060972 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4a750-73d4-4b46-a007-2846983da704 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:36.065446 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3144330d-32ee-42a1-8acb-45f57f9c808e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:36.066412 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144330d-32ee-42a1-8acb-45f57f9c808e admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:36.066756 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144330d-32ee-42a1-8acb-45f57f9c808e admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:36.067118 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144330d-32ee-42a1-8acb-45f57f9c808e admin admin] PATH_INFO: `/v3/projects/efe2965cb3f7439789a6326d50996001/users/d2a35ba0e5cb4e6bbff4e8eb8e32db6b/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:36.083618 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3144330d-32ee-42a1-8acb-45f57f9c808e admin admin] RBAC: Authorizing `identity:create_grant(project_id=efe2965cb3f7439789a6326d50996001, user_id=d2a35ba0e5cb4e6bbff4e8eb8e32db6b, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:36.086640 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3144330d-32ee-42a1-8acb-45f57f9c808e admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:36.101109 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1254/2495] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:55:36 2022] PUT /identity/v3/projects/efe2965cb3f7439789a6326d50996001/users/d2a35ba0e5cb4e6bbff4e8eb8e32db6b/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:55:36.105127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca5dbb-d041-42e0-b538-e9ec158cd5c9 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:36.116043 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca5dbb-d041-42e0-b538-e9ec158cd5c9 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:36.148528 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:55:36.179355 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d229449-5478-46c9-89fa-67bc66d089cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:36.180027 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d229449-5478-46c9-89fa-67bc66d089cb admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:36.180357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d229449-5478-46c9-89fa-67bc66d089cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:36.180637 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d229449-5478-46c9-89fa-67bc66d089cb admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:36.181398 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d229449-5478-46c9-89fa-67bc66d089cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:36.182988 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d229449-5478-46c9-89fa-67bc66d089cb admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:36.193576 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1242/2496] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:55:36.197996 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3144330d-32ee-42a1-8acb-45f57f9c808e admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:36.210010 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3144330d-32ee-42a1-8acb-45f57f9c808e admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:36.214589 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad11564-f430-4e8e-8a89-f31c1f9cf743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:36.215506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad11564-f430-4e8e-8a89-f31c1f9cf743 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:36.215852 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad11564-f430-4e8e-8a89-f31c1f9cf743 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:36.216225 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad11564-f430-4e8e-8a89-f31c1f9cf743 admin admin] PATH_INFO: `/v3/projects/efe2965cb3f7439789a6326d50996001/users/d2a35ba0e5cb4e6bbff4e8eb8e32db6b/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:36.218597 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad11564-f430-4e8e-8a89-f31c1f9cf743 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efe2965cb3f7439789a6326d50996001, user_id=d2a35ba0e5cb4e6bbff4e8eb8e32db6b, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:36.221332 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad11564-f430-4e8e-8a89-f31c1f9cf743 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:36.232469 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1255/2497] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:55:36 2022] PUT /identity/v3/projects/efe2965cb3f7439789a6326d50996001/users/d2a35ba0e5cb4e6bbff4e8eb8e32db6b/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:55:36.243820 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593cd75-a15a-4c0c-88b7-c2e1f5fca5d5 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:36.244695 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593cd75-a15a-4c0c-88b7-c2e1f5fca5d5 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:36.245065 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593cd75-a15a-4c0c-88b7-c2e1f5fca5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:36.275559 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-6593cd75-a15a-4c0c-88b7-c2e1f5fca5d5 None None] MFA Rules not processed for user `d2a35ba0e5cb4e6bbff4e8eb8e32db6b`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:55:36.276785 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-6593cd75-a15a-4c0c-88b7-c2e1f5fca5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:36.353486 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1243/2498] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:55:36 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:55:36.365155 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad11564-f430-4e8e-8a89-f31c1f9cf743 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:36.366887 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-9ad11564-f430-4e8e-8a89-f31c1f9cf743 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:36.379029 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:55:36.445321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad11564-f430-4e8e-8a89-f31c1f9cf743 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:36.448884 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2bd41-bb76-4c52-8fd0-f664164ba7b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:36.449605 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2bd41-bb76-4c52-8fd0-f664164ba7b6 service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:36.449887 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2bd41-bb76-4c52-8fd0-f664164ba7b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:36.450841 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2bd41-bb76-4c52-8fd0-f664164ba7b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:36.460737 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba2bd41-bb76-4c52-8fd0-f664164ba7b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:36.462207 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba2bd41-bb76-4c52-8fd0-f664164ba7b6 service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:36.473709 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1256/2499] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:55:36 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:55:36.909963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593cd75-a15a-4c0c-88b7-c2e1f5fca5d5 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:36.984500 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593cd75-a15a-4c0c-88b7-c2e1f5fca5d5 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:36.989461 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65b616-aa8d-4537-9e05-a75413a0b8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:36.990470 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65b616-aa8d-4537-9e05-a75413a0b8a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:36.990823 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65b616-aa8d-4537-9e05-a75413a0b8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:36.991204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65b616-aa8d-4537-9e05-a75413a0b8a7 admin admin] PATH_INFO: `/v3/users/5e232b13cfe84689ae7ee06aacb7f813` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:36.992560 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d65b616-aa8d-4537-9e05-a75413a0b8a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e232b13cfe84689ae7ee06aacb7f813)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:36.995005 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d65b616-aa8d-4537-9e05-a75413a0b8a7 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:37.048922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-9d65b616-aa8d-4537-9e05-a75413a0b8a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e232b13cfe84689ae7ee06aacb7f813'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:37.058592 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-9d65b616-aa8d-4537-9e05-a75413a0b8a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e232b13cfe84689ae7ee06aacb7f813'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:37.059398 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-9d65b616-aa8d-4537-9e05-a75413a0b8a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e232b13cfe84689ae7ee06aacb7f813'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:37.061479 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:55:37.073042 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-9d65b616-aa8d-4537-9e05-a75413a0b8a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e232b13cfe84689ae7ee06aacb7f813'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:37.088873 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-9d65b616-aa8d-4537-9e05-a75413a0b8a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e232b13cfe84689ae7ee06aacb7f813'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:37.096499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1244/2500] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:55:36 2022] DELETE /identity/v3/users/5e232b13cfe84689ae7ee06aacb7f813 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:55:37.100695 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac3e74-ab73-4934-b4a3-7e3890699904 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:37.101223 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac3e74-ab73-4934-b4a3-7e3890699904 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:37.101312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac3e74-ab73-4934-b4a3-7e3890699904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:37.119669 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-81ac3e74-ab73-4934-b4a3-7e3890699904 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:55:37.121307 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-81ac3e74-ab73-4934-b4a3-7e3890699904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:37.193080 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1257/2501] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:55:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:55:37.203498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65b616-aa8d-4537-9e05-a75413a0b8a7 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:37.204929 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-9d65b616-aa8d-4537-9e05-a75413a0b8a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:37.263424 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65b616-aa8d-4537-9e05-a75413a0b8a7 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:37.267126 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6703ee78-416a-4ba3-9a49-a757f903d92d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:37.267756 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703ee78-416a-4ba3-9a49-a757f903d92d service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:37.268040 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703ee78-416a-4ba3-9a49-a757f903d92d service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:37.268314 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703ee78-416a-4ba3-9a49-a757f903d92d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:37.277951 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6703ee78-416a-4ba3-9a49-a757f903d92d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:37.279982 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6703ee78-416a-4ba3-9a49-a757f903d92d service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:37.289735 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1245/2502] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:55:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:55:37.572468 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac3e74-ab73-4934-b4a3-7e3890699904 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:37.574071 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-81ac3e74-ab73-4934-b4a3-7e3890699904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:37.592825 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac3e74-ab73-4934-b4a3-7e3890699904 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:37.607425 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2962e-f08a-4cf8-83b0-b7b0a57f8a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:37.608307 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2962e-f08a-4cf8-83b0-b7b0a57f8a6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:37.608626 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2962e-f08a-4cf8-83b0-b7b0a57f8a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:37.608934 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2962e-f08a-4cf8-83b0-b7b0a57f8a6a admin admin] PATH_INFO: `/v3/projects/13d9ba29cdb34d11bf82fa510e7a7a29` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:37.610722 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd2962e-f08a-4cf8-83b0-b7b0a57f8a6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=13d9ba29cdb34d11bf82fa510e7a7a29)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:37.613046 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd2962e-f08a-4cf8-83b0-b7b0a57f8a6a admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:37.682265 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-7fd2962e-f08a-4cf8-83b0-b7b0a57f8a6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13d9ba29cdb34d11bf82fa510e7a7a29', 'request_id': 'req-7fd2962e-f08a-4cf8-83b0-b7b0a57f8a6a'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:37.690813 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-7fd2962e-f08a-4cf8-83b0-b7b0a57f8a6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13d9ba29cdb34d11bf82fa510e7a7a29', 'request_id': 'req-7fd2962e-f08a-4cf8-83b0-b7b0a57f8a6a'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:37.698637 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-7fd2962e-f08a-4cf8-83b0-b7b0a57f8a6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13d9ba29cdb34d11bf82fa510e7a7a29', 'request_id': 'req-7fd2962e-f08a-4cf8-83b0-b7b0a57f8a6a'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:37.712198 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-7fd2962e-f08a-4cf8-83b0-b7b0a57f8a6a admin admin] The token cache is being invalidate because project 13d9ba29cdb34d11bf82fa510e7a7a29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:55:37.712492 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-7fd2962e-f08a-4cf8-83b0-b7b0a57f8a6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:37.713192 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-7fd2962e-f08a-4cf8-83b0-b7b0a57f8a6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:37.714778 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1258/2503] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:55:37 2022] DELETE /identity/v3/projects/13d9ba29cdb34d11bf82fa510e7a7a29 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:55:37.719370 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6703ee78-416a-4ba3-9a49-a757f903d92d service neutron] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:37.730074 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6703ee78-416a-4ba3-9a49-a757f903d92d service neutron] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:37.798750 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf54e9-dcbf-4ecd-903b-8383dbe97ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:37.799897 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf54e9-dcbf-4ecd-903b-8383dbe97ce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:37.800187 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf54e9-dcbf-4ecd-903b-8383dbe97ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:37.800463 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf54e9-dcbf-4ecd-903b-8383dbe97ce8 admin admin] PATH_INFO: `/v3/users/d2a35ba0e5cb4e6bbff4e8eb8e32db6b` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:37.801760 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf54e9-dcbf-4ecd-903b-8383dbe97ce8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2a35ba0e5cb4e6bbff4e8eb8e32db6b)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:37.803739 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf54e9-dcbf-4ecd-903b-8383dbe97ce8 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:37.854980 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-dcdf54e9-dcbf-4ecd-903b-8383dbe97ce8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2a35ba0e5cb4e6bbff4e8eb8e32db6b'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:37.865116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-dcdf54e9-dcbf-4ecd-903b-8383dbe97ce8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2a35ba0e5cb4e6bbff4e8eb8e32db6b'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:37.865116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-dcdf54e9-dcbf-4ecd-903b-8383dbe97ce8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2a35ba0e5cb4e6bbff4e8eb8e32db6b'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:37.872136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-dcdf54e9-dcbf-4ecd-903b-8383dbe97ce8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2a35ba0e5cb4e6bbff4e8eb8e32db6b'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:37.883758 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:55:37.886722 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-dcdf54e9-dcbf-4ecd-903b-8383dbe97ce8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2a35ba0e5cb4e6bbff4e8eb8e32db6b'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:37.898244 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1246/2504] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:55:37 2022] DELETE /identity/v3/users/d2a35ba0e5cb4e6bbff4e8eb8e32db6b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:55:38.237278 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2962e-f08a-4cf8-83b0-b7b0a57f8a6a admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:38.239298 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-7fd2962e-f08a-4cf8-83b0-b7b0a57f8a6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:38.259033 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2962e-f08a-4cf8-83b0-b7b0a57f8a6a admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:38.296761 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:55:38.324777 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49b54b-b0b3-43bd-8d30-948c528730cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:38.326536 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49b54b-b0b3-43bd-8d30-948c528730cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:38.326817 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49b54b-b0b3-43bd-8d30-948c528730cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:38.327091 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49b54b-b0b3-43bd-8d30-948c528730cd admin admin] PATH_INFO: `/v3/projects/efe2965cb3f7439789a6326d50996001` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:38.328268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e49b54b-b0b3-43bd-8d30-948c528730cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=efe2965cb3f7439789a6326d50996001)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:38.330186 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e49b54b-b0b3-43bd-8d30-948c528730cd admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:38.376029 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-1e49b54b-b0b3-43bd-8d30-948c528730cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efe2965cb3f7439789a6326d50996001', 'request_id': 'req-1e49b54b-b0b3-43bd-8d30-948c528730cd'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:38.381666 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-1e49b54b-b0b3-43bd-8d30-948c528730cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efe2965cb3f7439789a6326d50996001', 'request_id': 'req-1e49b54b-b0b3-43bd-8d30-948c528730cd'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:38.389334 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-1e49b54b-b0b3-43bd-8d30-948c528730cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efe2965cb3f7439789a6326d50996001', 'request_id': 'req-1e49b54b-b0b3-43bd-8d30-948c528730cd'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:38.400302 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-1e49b54b-b0b3-43bd-8d30-948c528730cd admin admin] The token cache is being invalidate because project efe2965cb3f7439789a6326d50996001 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:55:38.400591 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-1e49b54b-b0b3-43bd-8d30-948c528730cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:38.401246 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-1e49b54b-b0b3-43bd-8d30-948c528730cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:38.402778 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1259/2505] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:55:38 2022] DELETE /identity/v3/projects/efe2965cb3f7439789a6326d50996001 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:55:38.407658 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baddc9c2-eb81-4c39-b3d3-2410fe2aa810 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:38.408129 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baddc9c2-eb81-4c39-b3d3-2410fe2aa810 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:38.408288 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baddc9c2-eb81-4c39-b3d3-2410fe2aa810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:38.426572 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-baddc9c2-eb81-4c39-b3d3-2410fe2aa810 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:55:38.427788 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-baddc9c2-eb81-4c39-b3d3-2410fe2aa810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:38.492676 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1247/2506] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:55:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:55:38.501489 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd736b7-9321-47a3-886e-79685a822388 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:38.501987 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd736b7-9321-47a3-886e-79685a822388 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:38.502431 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd736b7-9321-47a3-886e-79685a822388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:38.534024 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-1fd736b7-9321-47a3-886e-79685a822388 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:55:38.535136 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-1fd736b7-9321-47a3-886e-79685a822388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:38.546951 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1260/2507] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:55:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:55:38.548115 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baddc9c2-eb81-4c39-b3d3-2410fe2aa810 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:38.561875 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baddc9c2-eb81-4c39-b3d3-2410fe2aa810 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:38.566811 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ead28-c4f0-4d46-aba5-5c405b618ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:38.567471 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ead28-c4f0-4d46-aba5-5c405b618ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:38.567756 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ead28-c4f0-4d46-aba5-5c405b618ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:38.568031 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ead28-c4f0-4d46-aba5-5c405b618ef8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:38.568758 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85ead28-c4f0-4d46-aba5-5c405b618ef8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:55:38.569095 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85ead28-c4f0-4d46-aba5-5c405b618ef8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:38.570737 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85ead28-c4f0-4d46-aba5-5c405b618ef8 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:38.588463 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1248/2508] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:55:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:55:38.589726 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd736b7-9321-47a3-886e-79685a822388 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:38.599287 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd736b7-9321-47a3-886e-79685a822388 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:38.604111 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32e792b-a4c4-4a88-94d6-3bcc7e9097b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:38.604754 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e792b-a4c4-4a88-94d6-3bcc7e9097b8 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:38.605061 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e792b-a4c4-4a88-94d6-3bcc7e9097b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:38.605338 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e792b-a4c4-4a88-94d6-3bcc7e9097b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:38.606065 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32e792b-a4c4-4a88-94d6-3bcc7e9097b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:38.608624 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32e792b-a4c4-4a88-94d6-3bcc7e9097b8 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:38.635495 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1261/2509] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:55:38 2022] POST /identity/v3/projects => generated 407 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:55:38.639472 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ead28-c4f0-4d46-aba5-5c405b618ef8 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:38.649983 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ead28-c4f0-4d46-aba5-5c405b618ef8 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:38.709044 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe312c-6cc8-45e9-b281-e9618218711d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:38.709795 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe312c-6cc8-45e9-b281-e9618218711d admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:38.710153 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe312c-6cc8-45e9-b281-e9618218711d admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:38.710498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe312c-6cc8-45e9-b281-e9618218711d admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:38.711197 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe312c-6cc8-45e9-b281-e9618218711d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:38.713162 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe312c-6cc8-45e9-b281-e9618218711d admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:38.733751 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1249/2510] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:55:38 2022] POST /identity/v3/users => generated 304 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:55:38.737958 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32e792b-a4c4-4a88-94d6-3bcc7e9097b8 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:38.750823 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32e792b-a4c4-4a88-94d6-3bcc7e9097b8 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:38.755731 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6e1f24-aaa5-4672-86ea-eb84d55a5cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:38.756410 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e1f24-aaa5-4672-86ea-eb84d55a5cc5 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:38.756682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e1f24-aaa5-4672-86ea-eb84d55a5cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:38.756950 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e1f24-aaa5-4672-86ea-eb84d55a5cc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:38.757704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6e1f24-aaa5-4672-86ea-eb84d55a5cc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:38.759366 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6e1f24-aaa5-4672-86ea-eb84d55a5cc5 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:38.770678 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1262/2511] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:55:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:55:38.774692 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe312c-6cc8-45e9-b281-e9618218711d admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:38.785270 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe312c-6cc8-45e9-b281-e9618218711d admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:38.802931 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0676182-ce02-470a-a864-0b227f9f56aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:38.803901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0676182-ce02-470a-a864-0b227f9f56aa admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:38.804252 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0676182-ce02-470a-a864-0b227f9f56aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:38.804627 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0676182-ce02-470a-a864-0b227f9f56aa admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:38.805907 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0676182-ce02-470a-a864-0b227f9f56aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:38.807899 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0676182-ce02-470a-a864-0b227f9f56aa admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:38.824129 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1250/2512] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:55:38 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:55:38.825300 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6e1f24-aaa5-4672-86ea-eb84d55a5cc5 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:38.838599 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6e1f24-aaa5-4672-86ea-eb84d55a5cc5 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:38.846464 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b665e-515a-418e-b3f4-efb130a3eb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:38.847624 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b665e-515a-418e-b3f4-efb130a3eb08 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:38.848068 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b665e-515a-418e-b3f4-efb130a3eb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:38.848505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b665e-515a-418e-b3f4-efb130a3eb08 admin admin] PATH_INFO: `/v3/projects/9a63187860c94c1e9cb403b389f6c268/users/c9680173a8004702b547a6348569abde/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:38.871645 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0b665e-515a-418e-b3f4-efb130a3eb08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a63187860c94c1e9cb403b389f6c268, user_id=c9680173a8004702b547a6348569abde, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:38.876593 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0b665e-515a-418e-b3f4-efb130a3eb08 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:38.894038 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1263/2513] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:55:38 2022] PUT /identity/v3/projects/9a63187860c94c1e9cb403b389f6c268/users/c9680173a8004702b547a6348569abde/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:55:38.905503 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24275ef-fdd7-4372-9900-4171c53ae252 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:38.905815 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24275ef-fdd7-4372-9900-4171c53ae252 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:38.906077 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24275ef-fdd7-4372-9900-4171c53ae252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:38.934227 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-d24275ef-fdd7-4372-9900-4171c53ae252 None None] MFA Rules not processed for user `c9680173a8004702b547a6348569abde`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:55:38.936821 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-d24275ef-fdd7-4372-9900-4171c53ae252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:38.988575 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:55:39.010354 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1251/2514] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:55:38 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:55:39.014865 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b665e-515a-418e-b3f4-efb130a3eb08 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:39.097280 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b665e-515a-418e-b3f4-efb130a3eb08 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:39.103516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6fca72-8c1f-42e6-a017-a13bf344220f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:39.104178 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6fca72-8c1f-42e6-a017-a13bf344220f admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:39.104455 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6fca72-8c1f-42e6-a017-a13bf344220f admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:39.104891 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6fca72-8c1f-42e6-a017-a13bf344220f admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:39.105647 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6fca72-8c1f-42e6-a017-a13bf344220f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:39.109094 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6fca72-8c1f-42e6-a017-a13bf344220f admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:39.113551 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:55:39.146032 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1264/2515] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:55:39 2022] POST /identity/v3/projects => generated 409 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:55:39.150283 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24275ef-fdd7-4372-9900-4171c53ae252 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:39.166338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24275ef-fdd7-4372-9900-4171c53ae252 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:39.282620 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a5191-1bd3-42eb-8740-1bcbea844b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:39.283311 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a5191-1bd3-42eb-8740-1bcbea844b4c admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:39.283597 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a5191-1bd3-42eb-8740-1bcbea844b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:39.283873 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a5191-1bd3-42eb-8740-1bcbea844b4c admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:39.284609 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a5191-1bd3-42eb-8740-1bcbea844b4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:39.286516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a5191-1bd3-42eb-8740-1bcbea844b4c admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:39.334432 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1252/2516] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:55:39 2022] POST /identity/v3/users => generated 311 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:55:39.339844 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6fca72-8c1f-42e6-a017-a13bf344220f admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:39.352945 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6fca72-8c1f-42e6-a017-a13bf344220f admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:39.369705 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1c318-771e-4d20-8ad9-dde57c4da543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:39.370365 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1c318-771e-4d20-8ad9-dde57c4da543 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:39.370561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1c318-771e-4d20-8ad9-dde57c4da543 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:39.371001 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1c318-771e-4d20-8ad9-dde57c4da543 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:39.371579 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc1c318-771e-4d20-8ad9-dde57c4da543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:39.373926 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc1c318-771e-4d20-8ad9-dde57c4da543 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:39.386106 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1265/2517] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:55:39 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:55:39.390995 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a5191-1bd3-42eb-8740-1bcbea844b4c admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:39.407213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a5191-1bd3-42eb-8740-1bcbea844b4c admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:39.413861 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f51de8c-36e0-46cb-a195-d686056f5494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:39.416076 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51de8c-36e0-46cb-a195-d686056f5494 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:39.416688 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51de8c-36e0-46cb-a195-d686056f5494 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:39.417659 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51de8c-36e0-46cb-a195-d686056f5494 admin admin] PATH_INFO: `/v3/projects/8199376cd84a4c95ae6b94cfeb032886/users/57137a1598d043e29e8a9a85db47307c/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:39.438670 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f51de8c-36e0-46cb-a195-d686056f5494 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8199376cd84a4c95ae6b94cfeb032886, user_id=57137a1598d043e29e8a9a85db47307c, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:39.441745 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f51de8c-36e0-46cb-a195-d686056f5494 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:39.458535 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1253/2518] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:55:39 2022] PUT /identity/v3/projects/8199376cd84a4c95ae6b94cfeb032886/users/57137a1598d043e29e8a9a85db47307c/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:55:39.462361 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1c318-771e-4d20-8ad9-dde57c4da543 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:39.474447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1c318-771e-4d20-8ad9-dde57c4da543 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:39.560216 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf30171-c5af-400d-b3df-25cd1ec9b79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:39.561168 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf30171-c5af-400d-b3df-25cd1ec9b79f admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:39.561555 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf30171-c5af-400d-b3df-25cd1ec9b79f admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:39.561958 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf30171-c5af-400d-b3df-25cd1ec9b79f admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:39.563578 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf30171-c5af-400d-b3df-25cd1ec9b79f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:39.565227 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf30171-c5af-400d-b3df-25cd1ec9b79f admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:39.580178 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1266/2519] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:55:39 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:55:39.581381 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f51de8c-36e0-46cb-a195-d686056f5494 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:39.592872 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f51de8c-36e0-46cb-a195-d686056f5494 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:39.599679 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8d4e88-bf15-4830-8ecb-d1d665d4c22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:39.600819 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d4e88-bf15-4830-8ecb-d1d665d4c22e admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:39.601320 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d4e88-bf15-4830-8ecb-d1d665d4c22e admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:39.601703 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d4e88-bf15-4830-8ecb-d1d665d4c22e admin admin] PATH_INFO: `/v3/projects/8199376cd84a4c95ae6b94cfeb032886/users/57137a1598d043e29e8a9a85db47307c/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:39.603858 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8d4e88-bf15-4830-8ecb-d1d665d4c22e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8199376cd84a4c95ae6b94cfeb032886, user_id=57137a1598d043e29e8a9a85db47307c, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:39.607767 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8d4e88-bf15-4830-8ecb-d1d665d4c22e admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:39.619014 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1254/2520] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:55:39 2022] PUT /identity/v3/projects/8199376cd84a4c95ae6b94cfeb032886/users/57137a1598d043e29e8a9a85db47307c/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:55:39.631401 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4bd500-91d6-467e-bdcb-0dcfc14e82fd None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:39.631690 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4bd500-91d6-467e-bdcb-0dcfc14e82fd None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:39.631972 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4bd500-91d6-467e-bdcb-0dcfc14e82fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:39.678854 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-bc4bd500-91d6-467e-bdcb-0dcfc14e82fd None None] MFA Rules not processed for user `57137a1598d043e29e8a9a85db47307c`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:55:39.680671 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-bc4bd500-91d6-467e-bdcb-0dcfc14e82fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:39.695280 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754bb93-695b-4c1c-a9a9-72036c435a70 None None] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:39.695595 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754bb93-695b-4c1c-a9a9-72036c435a70 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:39.695874 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754bb93-695b-4c1c-a9a9-72036c435a70 None None] PATH_INFO: `/` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:39.697294 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1255/2521] 10.222.0.20 () {54 vars in 921 bytes} [Wed Jul 6 13:55:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 13:55:39.703685 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa1484-5628-4841-8a50-164dd9cddbe8 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:39.703972 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa1484-5628-4841-8a50-164dd9cddbe8 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:39.704261 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa1484-5628-4841-8a50-164dd9cddbe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:39.754196 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-fcaa1484-5628-4841-8a50-164dd9cddbe8 None None] MFA Rules not processed for user `6647eb1e8a1544ef9d3a9806f2d1fc51`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:55:39.755452 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-fcaa1484-5628-4841-8a50-164dd9cddbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:39.775671 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1267/2522] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:55:39 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:55:39.791758 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4bd500-91d6-467e-bdcb-0dcfc14e82fd None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:39.795567 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-bc4bd500-91d6-467e-bdcb-0dcfc14e82fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:39.852580 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1256/2523] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Jul 6 13:55:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:55:39.866015 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa1484-5628-4841-8a50-164dd9cddbe8 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:39.873613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-fcaa1484-5628-4841-8a50-164dd9cddbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:39.906125 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4bd500-91d6-467e-bdcb-0dcfc14e82fd None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:39.911023 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5727352-bcb3-48f1-96f2-7dedd30e8310 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:39.911777 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5727352-bcb3-48f1-96f2-7dedd30e8310 service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:39.912116 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5727352-bcb3-48f1-96f2-7dedd30e8310 service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:39.912534 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5727352-bcb3-48f1-96f2-7dedd30e8310 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:39.913049 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:55:39.929088 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5727352-bcb3-48f1-96f2-7dedd30e8310 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:39.931120 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5727352-bcb3-48f1-96f2-7dedd30e8310 service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:39.937810 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:55:39.945959 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa1484-5628-4841-8a50-164dd9cddbe8 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:39.949895 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1268/2524] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:55:39 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:55:39.951672 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b7e283-a18b-4729-ab37-0d962cf81dbc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4015c5ca2b8a438288820cd003f9a308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:39.952289 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7e283-a18b-4729-ab37-0d962cf81dbc service glance] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:39.952995 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7e283-a18b-4729-ab37-0d962cf81dbc service glance] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:39.953325 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7e283-a18b-4729-ab37-0d962cf81dbc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:39.967278 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b7e283-a18b-4729-ab37-0d962cf81dbc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:39.968868 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b7e283-a18b-4729-ab37-0d962cf81dbc service glance] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:39.984257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1257/2525] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:55:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 13:55:40.106537 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5727352-bcb3-48f1-96f2-7dedd30e8310 service cinder] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:40.116373 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5727352-bcb3-48f1-96f2-7dedd30e8310 service cinder] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:40.120053 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2fb6e4-c700-4207-97fc-e842fa161790 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:40.120655 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2fb6e4-c700-4207-97fc-e842fa161790 service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:40.120937 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2fb6e4-c700-4207-97fc-e842fa161790 service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:40.121239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2fb6e4-c700-4207-97fc-e842fa161790 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:40.179994 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2fb6e4-c700-4207-97fc-e842fa161790 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:40.181771 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2fb6e4-c700-4207-97fc-e842fa161790 service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:40.191657 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1269/2526] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:55:40 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:55:46.550503 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4fbdf-63ea-471f-905d-1db86ebcf01e None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:46.550989 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4fbdf-63ea-471f-905d-1db86ebcf01e None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:46.551105 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4fbdf-63ea-471f-905d-1db86ebcf01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:46.568518 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-acb4fbdf-63ea-471f-905d-1db86ebcf01e None None] MFA Rules not processed for user `c4535134181e489fb532f617264d5855`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:55:46.569741 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-acb4fbdf-63ea-471f-905d-1db86ebcf01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:46.624414 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1258/2527] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Jul 6 13:55:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:55:46.636255 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2fb6e4-c700-4207-97fc-e842fa161790 service cinder] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:46.637695 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-fa2fb6e4-c700-4207-97fc-e842fa161790 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:46.703162 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4fbdf-63ea-471f-905d-1db86ebcf01e None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:46.704631 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-acb4fbdf-63ea-471f-905d-1db86ebcf01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:46.712182 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2fb6e4-c700-4207-97fc-e842fa161790 service cinder] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:46.717645 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed0fc44-3458-40ef-872c-2cedc9459b59 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c1bea05c37646b0a05fa8dda0367ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:46.718290 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0fc44-3458-40ef-872c-2cedc9459b59 service swift] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:46.718569 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0fc44-3458-40ef-872c-2cedc9459b59 service swift] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:46.718850 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0fc44-3458-40ef-872c-2cedc9459b59 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:46.726826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed0fc44-3458-40ef-872c-2cedc9459b59 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:46.728382 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed0fc44-3458-40ef-872c-2cedc9459b59 service swift] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:46.736838 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1270/2528] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jul 6 13:55:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 13:55:46.764163 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4fbdf-63ea-471f-905d-1db86ebcf01e None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:46.767948 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769374c2-0080-4e95-ac4c-a10c965d22aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:46.768447 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769374c2-0080-4e95-ac4c-a10c965d22aa service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:46.768733 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769374c2-0080-4e95-ac4c-a10c965d22aa service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:46.769030 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769374c2-0080-4e95-ac4c-a10c965d22aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:46.770803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-769374c2-0080-4e95-ac4c-a10c965d22aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:46.914856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769374c2-0080-4e95-ac4c-a10c965d22aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:46.936026 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769374c2-0080-4e95-ac4c-a10c965d22aa service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:46.973376 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1259/2529] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:55:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 13:55:49.935929 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed0fc44-3458-40ef-872c-2cedc9459b59 service swift] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:49.958327 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed0fc44-3458-40ef-872c-2cedc9459b59 service swift] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:49.971596 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b5ba06-759e-4c6a-a38c-e88f5da6d58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:49.972546 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5ba06-759e-4c6a-a38c-e88f5da6d58c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:49.972918 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5ba06-759e-4c6a-a38c-e88f5da6d58c admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:49.973289 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5ba06-759e-4c6a-a38c-e88f5da6d58c admin admin] PATH_INFO: `/v3/users/c9680173a8004702b547a6348569abde` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:49.976389 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b5ba06-759e-4c6a-a38c-e88f5da6d58c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9680173a8004702b547a6348569abde)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:49.980788 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b5ba06-759e-4c6a-a38c-e88f5da6d58c admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:50.091583 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-a3b5ba06-759e-4c6a-a38c-e88f5da6d58c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9680173a8004702b547a6348569abde'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:50.111733 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-a3b5ba06-759e-4c6a-a38c-e88f5da6d58c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9680173a8004702b547a6348569abde'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:50.128630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-a3b5ba06-759e-4c6a-a38c-e88f5da6d58c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9680173a8004702b547a6348569abde'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:50.129556 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-a3b5ba06-759e-4c6a-a38c-e88f5da6d58c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9680173a8004702b547a6348569abde'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:50.151239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-a3b5ba06-759e-4c6a-a38c-e88f5da6d58c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9680173a8004702b547a6348569abde'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:50.155115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:55:50.161768 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1271/2530] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:55:49 2022] DELETE /identity/v3/users/c9680173a8004702b547a6348569abde => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:55:50.165485 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22311457-9a36-4a9a-91fb-399b689aaecf None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:50.166342 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22311457-9a36-4a9a-91fb-399b689aaecf None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:50.166793 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22311457-9a36-4a9a-91fb-399b689aaecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:50.199140 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-22311457-9a36-4a9a-91fb-399b689aaecf None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:55:50.202735 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-22311457-9a36-4a9a-91fb-399b689aaecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:50.239867 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:55:50.293070 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1260/2531] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:55:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:55:50.303608 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b5ba06-759e-4c6a-a38c-e88f5da6d58c admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:50.305083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-a3b5ba06-759e-4c6a-a38c-e88f5da6d58c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:50.433061 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b5ba06-759e-4c6a-a38c-e88f5da6d58c admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:50.438836 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36a591-4c74-415e-9688-3566b9edcdd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:50.445070 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36a591-4c74-415e-9688-3566b9edcdd4 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:50.449965 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36a591-4c74-415e-9688-3566b9edcdd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:50.451215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36a591-4c74-415e-9688-3566b9edcdd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:50.466003 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf36a591-4c74-415e-9688-3566b9edcdd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:50.467618 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf36a591-4c74-415e-9688-3566b9edcdd4 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:50.489519 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1272/2532] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:55:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 13:55:50.901110 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22311457-9a36-4a9a-91fb-399b689aaecf None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:50.902929 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-22311457-9a36-4a9a-91fb-399b689aaecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:50.929767 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22311457-9a36-4a9a-91fb-399b689aaecf None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:50.939422 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48fb8a-09ef-409e-89cd-bdb3b41290b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:50.940253 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48fb8a-09ef-409e-89cd-bdb3b41290b7 service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:50.941575 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48fb8a-09ef-409e-89cd-bdb3b41290b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:50.942631 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48fb8a-09ef-409e-89cd-bdb3b41290b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:50.946570 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-8a48fb8a-09ef-409e-89cd-bdb3b41290b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:51.005793 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36a591-4c74-415e-9688-3566b9edcdd4 service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:51.007558 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-bf36a591-4c74-415e-9688-3566b9edcdd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:51.022678 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36a591-4c74-415e-9688-3566b9edcdd4 service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:51.029084 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f45da0-9e1b-48ce-8423-1a5b000aab31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:51.029949 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f45da0-9e1b-48ce-8423-1a5b000aab31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:51.030559 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f45da0-9e1b-48ce-8423-1a5b000aab31 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:51.030841 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f45da0-9e1b-48ce-8423-1a5b000aab31 admin admin] PATH_INFO: `/v3/projects/9a63187860c94c1e9cb403b389f6c268` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:51.051575 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f45da0-9e1b-48ce-8423-1a5b000aab31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a63187860c94c1e9cb403b389f6c268)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:51.056245 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f45da0-9e1b-48ce-8423-1a5b000aab31 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:51.241309 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a48fb8a-09ef-409e-89cd-bdb3b41290b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:51.242998 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a48fb8a-09ef-409e-89cd-bdb3b41290b7 service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:51.273399 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1261/2533] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:55:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:55:51.291121 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-89f45da0-9e1b-48ce-8423-1a5b000aab31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a63187860c94c1e9cb403b389f6c268', 'request_id': 'req-89f45da0-9e1b-48ce-8423-1a5b000aab31'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:51.300949 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-89f45da0-9e1b-48ce-8423-1a5b000aab31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a63187860c94c1e9cb403b389f6c268', 'request_id': 'req-89f45da0-9e1b-48ce-8423-1a5b000aab31'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:51.324035 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-89f45da0-9e1b-48ce-8423-1a5b000aab31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a63187860c94c1e9cb403b389f6c268', 'request_id': 'req-89f45da0-9e1b-48ce-8423-1a5b000aab31'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:51.346840 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-89f45da0-9e1b-48ce-8423-1a5b000aab31 admin admin] The token cache is being invalidate because project 9a63187860c94c1e9cb403b389f6c268 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:55:51.347403 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-89f45da0-9e1b-48ce-8423-1a5b000aab31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:51.348248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-89f45da0-9e1b-48ce-8423-1a5b000aab31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:51.350413 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1273/2534] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:55:51 2022] DELETE /identity/v3/projects/9a63187860c94c1e9cb403b389f6c268 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:55:51.354984 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48fb8a-09ef-409e-89cd-bdb3b41290b7 service neutron] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:51.373455 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48fb8a-09ef-409e-89cd-bdb3b41290b7 service neutron] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:51.486271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f06e5a-4346-4c77-aaff-9eead78c9da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:51.487172 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f06e5a-4346-4c77-aaff-9eead78c9da4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:51.487453 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f06e5a-4346-4c77-aaff-9eead78c9da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:51.487727 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f06e5a-4346-4c77-aaff-9eead78c9da4 admin admin] PATH_INFO: `/v3/users/57137a1598d043e29e8a9a85db47307c` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:51.488967 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f06e5a-4346-4c77-aaff-9eead78c9da4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57137a1598d043e29e8a9a85db47307c)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:51.492730 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f06e5a-4346-4c77-aaff-9eead78c9da4 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:51.579345 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-92f06e5a-4346-4c77-aaff-9eead78c9da4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57137a1598d043e29e8a9a85db47307c'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:51.589382 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-92f06e5a-4346-4c77-aaff-9eead78c9da4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57137a1598d043e29e8a9a85db47307c'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:51.590140 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-92f06e5a-4346-4c77-aaff-9eead78c9da4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57137a1598d043e29e8a9a85db47307c'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:51.596328 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:55:51.601749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-92f06e5a-4346-4c77-aaff-9eead78c9da4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57137a1598d043e29e8a9a85db47307c'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:51.619076 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-92f06e5a-4346-4c77-aaff-9eead78c9da4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57137a1598d043e29e8a9a85db47307c'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:51.626021 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1262/2535] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:55:51 2022] DELETE /identity/v3/users/57137a1598d043e29e8a9a85db47307c => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:55:51.745399 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f45da0-9e1b-48ce-8423-1a5b000aab31 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:51.748496 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-89f45da0-9e1b-48ce-8423-1a5b000aab31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:51.779937 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f45da0-9e1b-48ce-8423-1a5b000aab31 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:51.893034 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b82e49-f2ce-4000-a002-225e397f3a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:51.893957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b82e49-f2ce-4000-a002-225e397f3a8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:51.894236 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b82e49-f2ce-4000-a002-225e397f3a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:51.894514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b82e49-f2ce-4000-a002-225e397f3a8d admin admin] PATH_INFO: `/v3/users/7b572a8f465048558fc969b85b2aec5b` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:51.912464 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b82e49-f2ce-4000-a002-225e397f3a8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b572a8f465048558fc969b85b2aec5b)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:51.915309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b82e49-f2ce-4000-a002-225e397f3a8d admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:51.934486 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:55:51.999879 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c0b82e49-f2ce-4000-a002-225e397f3a8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b572a8f465048558fc969b85b2aec5b'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:52.014306 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c0b82e49-f2ce-4000-a002-225e397f3a8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b572a8f465048558fc969b85b2aec5b'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:52.024916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c0b82e49-f2ce-4000-a002-225e397f3a8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b572a8f465048558fc969b85b2aec5b'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:52.026377 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c0b82e49-f2ce-4000-a002-225e397f3a8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b572a8f465048558fc969b85b2aec5b'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:52.044065 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-c0b82e49-f2ce-4000-a002-225e397f3a8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b572a8f465048558fc969b85b2aec5b'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:52.063243 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1274/2536] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:55:51 2022] DELETE /identity/v3/users/7b572a8f465048558fc969b85b2aec5b => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:55:52.101525 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f06e5a-4346-4c77-aaff-9eead78c9da4 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:52.103292 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-92f06e5a-4346-4c77-aaff-9eead78c9da4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:52.125485 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f06e5a-4346-4c77-aaff-9eead78c9da4 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:52.228286 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a333db89-1324-4b30-aabb-936b01d37d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:52.229237 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333db89-1324-4b30-aabb-936b01d37d98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:52.229523 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333db89-1324-4b30-aabb-936b01d37d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:52.229801 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333db89-1324-4b30-aabb-936b01d37d98 admin admin] PATH_INFO: `/v3/projects/8199376cd84a4c95ae6b94cfeb032886` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:52.231677 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a333db89-1324-4b30-aabb-936b01d37d98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8199376cd84a4c95ae6b94cfeb032886)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:52.233817 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a333db89-1324-4b30-aabb-936b01d37d98 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:52.288408 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a333db89-1324-4b30-aabb-936b01d37d98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8199376cd84a4c95ae6b94cfeb032886', 'request_id': 'req-a333db89-1324-4b30-aabb-936b01d37d98'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:52.299697 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a333db89-1324-4b30-aabb-936b01d37d98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8199376cd84a4c95ae6b94cfeb032886', 'request_id': 'req-a333db89-1324-4b30-aabb-936b01d37d98'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:52.313033 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a333db89-1324-4b30-aabb-936b01d37d98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8199376cd84a4c95ae6b94cfeb032886', 'request_id': 'req-a333db89-1324-4b30-aabb-936b01d37d98'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:52.339306 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a333db89-1324-4b30-aabb-936b01d37d98 admin admin] The token cache is being invalidate because project 8199376cd84a4c95ae6b94cfeb032886 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:55:52.340073 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a333db89-1324-4b30-aabb-936b01d37d98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:52.341691 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a333db89-1324-4b30-aabb-936b01d37d98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:52.343970 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1263/2537] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:55:52 2022] DELETE /identity/v3/projects/8199376cd84a4c95ae6b94cfeb032886 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:55:52.349062 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86eadb-2119-40d7-8a4e-7bf51dfda002 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:52.349358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86eadb-2119-40d7-8a4e-7bf51dfda002 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:52.349715 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86eadb-2119-40d7-8a4e-7bf51dfda002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:52.377570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-7d86eadb-2119-40d7-8a4e-7bf51dfda002 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:55:52.378951 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-7d86eadb-2119-40d7-8a4e-7bf51dfda002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:52.380801 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a333db89-1324-4b30-aabb-936b01d37d98 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:52.382436 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a333db89-1324-4b30-aabb-936b01d37d98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:52.493150 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a333db89-1324-4b30-aabb-936b01d37d98 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:52.502584 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49bf2dd-20d2-418f-9c8a-b497b61754fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:52.503929 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49bf2dd-20d2-418f-9c8a-b497b61754fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:52.504264 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49bf2dd-20d2-418f-9c8a-b497b61754fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:52.504600 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49bf2dd-20d2-418f-9c8a-b497b61754fb admin admin] PATH_INFO: `/v3/projects/bd85f2a57a0847debf5aaf81aab7a638` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:52.517139 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1275/2538] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:55:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:55:52.518033 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:55:52.518838 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71a7df-d6a4-46c2-a039-cb66b6c8f88d None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:52.519221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71a7df-d6a4-46c2-a039-cb66b6c8f88d None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:52.519587 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71a7df-d6a4-46c2-a039-cb66b6c8f88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:52.522622 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49bf2dd-20d2-418f-9c8a-b497b61754fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd85f2a57a0847debf5aaf81aab7a638)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:52.526251 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49bf2dd-20d2-418f-9c8a-b497b61754fb admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:52.539488 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-0e71a7df-d6a4-46c2-a039-cb66b6c8f88d None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:55:52.540710 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-0e71a7df-d6a4-46c2-a039-cb66b6c8f88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:52.554616 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1276/2539] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:55:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:55:52.555643 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e71a7df-d6a4-46c2-a039-cb66b6c8f88d None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:52.566046 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e71a7df-d6a4-46c2-a039-cb66b6c8f88d None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:52.571616 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755d6107-db51-4c56-9a4c-f8cb086da9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:52.572176 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d6107-db51-4c56-9a4c-f8cb086da9d5 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:52.572538 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d6107-db51-4c56-9a4c-f8cb086da9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:52.572908 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d6107-db51-4c56-9a4c-f8cb086da9d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:52.573960 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755d6107-db51-4c56-9a4c-f8cb086da9d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:55:52.574314 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755d6107-db51-4c56-9a4c-f8cb086da9d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:52.576800 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755d6107-db51-4c56-9a4c-f8cb086da9d5 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:52.594471 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1277/2540] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:55:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:55:52.595454 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755d6107-db51-4c56-9a4c-f8cb086da9d5 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:52.602194 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-b49bf2dd-20d2-418f-9c8a-b497b61754fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd85f2a57a0847debf5aaf81aab7a638', 'request_id': 'req-b49bf2dd-20d2-418f-9c8a-b497b61754fb'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:52.606729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755d6107-db51-4c56-9a4c-f8cb086da9d5 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:52.612842 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-b49bf2dd-20d2-418f-9c8a-b497b61754fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd85f2a57a0847debf5aaf81aab7a638', 'request_id': 'req-b49bf2dd-20d2-418f-9c8a-b497b61754fb'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:52.624255 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-b49bf2dd-20d2-418f-9c8a-b497b61754fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd85f2a57a0847debf5aaf81aab7a638', 'request_id': 'req-b49bf2dd-20d2-418f-9c8a-b497b61754fb'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:52.643541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-b49bf2dd-20d2-418f-9c8a-b497b61754fb admin admin] The token cache is being invalidate because project bd85f2a57a0847debf5aaf81aab7a638 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:55:52.644617 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-b49bf2dd-20d2-418f-9c8a-b497b61754fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:52.645611 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-b49bf2dd-20d2-418f-9c8a-b497b61754fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:52.647811 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1264/2541] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:55:52 2022] DELETE /identity/v3/projects/bd85f2a57a0847debf5aaf81aab7a638 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:55:52.652924 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf00e1-35f9-4e7f-892d-e0d9205c55f7 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:52.654492 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf00e1-35f9-4e7f-892d-e0d9205c55f7 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:52.654492 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf00e1-35f9-4e7f-892d-e0d9205c55f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:52.686389 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0834dfc5-6b68-4b53-af97-c5d07b7cf438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:52.686659 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-45bf00e1-35f9-4e7f-892d-e0d9205c55f7 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:55:52.687299 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834dfc5-6b68-4b53-af97-c5d07b7cf438 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:52.687791 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834dfc5-6b68-4b53-af97-c5d07b7cf438 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:52.688247 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834dfc5-6b68-4b53-af97-c5d07b7cf438 admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:52.689093 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0834dfc5-6b68-4b53-af97-c5d07b7cf438 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:52.689618 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-45bf00e1-35f9-4e7f-892d-e0d9205c55f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:52.691561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0834dfc5-6b68-4b53-af97-c5d07b7cf438 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:52.713534 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1265/2542] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:55:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:55:52.720502 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b09a5-4e8d-4e56-9dcc-cdb2a5e3a8ab None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:52.721841 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b09a5-4e8d-4e56-9dcc-cdb2a5e3a8ab None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:52.722113 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b09a5-4e8d-4e56-9dcc-cdb2a5e3a8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:52.722825 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:55:52.740320 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1278/2543] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:55:52 2022] POST /identity/v3/projects => generated 371 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:55:52.743372 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0834dfc5-6b68-4b53-af97-c5d07b7cf438 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:52.749323 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-0834dfc5-6b68-4b53-af97-c5d07b7cf438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:52.759355 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-186b09a5-4e8d-4e56-9dcc-cdb2a5e3a8ab None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:55:52.760749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-186b09a5-4e8d-4e56-9dcc-cdb2a5e3a8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:52.872688 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0834dfc5-6b68-4b53-af97-c5d07b7cf438 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:52.880576 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1266/2544] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:55:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:55:52.890803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b09a5-4e8d-4e56-9dcc-cdb2a5e3a8ab None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:52.897239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaac918-45f1-4397-a36d-ef9d80cfea13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:52.905969 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaac918-45f1-4397-a36d-ef9d80cfea13 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:52.905969 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaac918-45f1-4397-a36d-ef9d80cfea13 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:52.905969 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaac918-45f1-4397-a36d-ef9d80cfea13 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:52.905969 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaac918-45f1-4397-a36d-ef9d80cfea13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:52.909087 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaac918-45f1-4397-a36d-ef9d80cfea13 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:52.916709 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b09a5-4e8d-4e56-9dcc-cdb2a5e3a8ab None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:52.930013 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac582a9-d7cc-49ea-bd2a-b11fd9196905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:52.932395 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac582a9-d7cc-49ea-bd2a-b11fd9196905 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:52.934169 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac582a9-d7cc-49ea-bd2a-b11fd9196905 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:52.935226 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac582a9-d7cc-49ea-bd2a-b11fd9196905 admin admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:52.936008 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac582a9-d7cc-49ea-bd2a-b11fd9196905 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:55:52.936325 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac582a9-d7cc-49ea-bd2a-b11fd9196905 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:52.938064 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac582a9-d7cc-49ea-bd2a-b11fd9196905 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:52.965140 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1279/2545] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:55:52 2022] POST /identity/v3/users => generated 286 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:55:52.967042 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaac918-45f1-4397-a36d-ef9d80cfea13 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:52.970074 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1267/2546] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:55:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:55:52.971142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac582a9-d7cc-49ea-bd2a-b11fd9196905 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:52.974200 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-ecaac918-45f1-4397-a36d-ef9d80cfea13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:52.986055 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac582a9-d7cc-49ea-bd2a-b11fd9196905 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:53.001987 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38041331-c558-465f-8e2c-7dd9e2ee981e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:53.003134 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38041331-c558-465f-8e2c-7dd9e2ee981e admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:53.003669 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38041331-c558-465f-8e2c-7dd9e2ee981e admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:53.004104 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38041331-c558-465f-8e2c-7dd9e2ee981e admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:53.005224 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38041331-c558-465f-8e2c-7dd9e2ee981e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:53.007694 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38041331-c558-465f-8e2c-7dd9e2ee981e admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:53.028064 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1268/2547] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:55:53.028064 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38041331-c558-465f-8e2c-7dd9e2ee981e admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:53.047569 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38041331-c558-465f-8e2c-7dd9e2ee981e admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:53.063817 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935fda5-687c-4626-a57b-045bec3341ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:53.064408 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935fda5-687c-4626-a57b-045bec3341ac admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:53.064697 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935fda5-687c-4626-a57b-045bec3341ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:53.064983 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935fda5-687c-4626-a57b-045bec3341ac admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:53.065707 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1935fda5-687c-4626-a57b-045bec3341ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:53.067658 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1935fda5-687c-4626-a57b-045bec3341ac admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:53.123607 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaac918-45f1-4397-a36d-ef9d80cfea13 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:53.133723 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ec56b-eddf-4fea-966f-775bc362a21d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:53.134403 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ec56b-eddf-4fea-966f-775bc362a21d service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:53.134676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ec56b-eddf-4fea-966f-775bc362a21d service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:53.134956 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ec56b-eddf-4fea-966f-775bc362a21d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:53.141184 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1269/2548] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:55:53 2022] POST /identity/v3/projects => generated 391 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:55:53.145558 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-c60ec56b-eddf-4fea-966f-775bc362a21d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:53.148633 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935fda5-687c-4626-a57b-045bec3341ac admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:53.232023 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:55:53.268040 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935fda5-687c-4626-a57b-045bec3341ac admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:53.269990 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60ec56b-eddf-4fea-966f-775bc362a21d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:53.271491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60ec56b-eddf-4fea-966f-775bc362a21d service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:53.273340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70182858-7b2c-485d-8df2-45d298c968ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:53.274990 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70182858-7b2c-485d-8df2-45d298c968ef admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:53.274990 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70182858-7b2c-485d-8df2-45d298c968ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:53.274990 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70182858-7b2c-485d-8df2-45d298c968ef admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:53.275344 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70182858-7b2c-485d-8df2-45d298c968ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:53.276890 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70182858-7b2c-485d-8df2-45d298c968ef admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:53.293132 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1270/2549] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:55:53.295098 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70182858-7b2c-485d-8df2-45d298c968ef admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:53.298696 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1280/2550] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:55:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:55:53.300133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ec56b-eddf-4fea-966f-775bc362a21d service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:53.321483 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70182858-7b2c-485d-8df2-45d298c968ef admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:53.326129 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ec56b-eddf-4fea-966f-775bc362a21d service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:53.333311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79532f31-bf62-4c95-866f-c20243b90e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:53.333311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79532f31-bf62-4c95-866f-c20243b90e87 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:53.333311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79532f31-bf62-4c95-866f-c20243b90e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:53.333781 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79532f31-bf62-4c95-866f-c20243b90e87 admin admin] PATH_INFO: `/v3/projects/dc655b1bf30b41aaa9c2de2b2db0cdd2/users/e6047b7680b745a1a8972fe7fdb4a1f6/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:53.336932 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8b4ff-9c73-4ed2-b5fb-7948297c14b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:53.337866 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8b4ff-9c73-4ed2-b5fb-7948297c14b8 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:53.345960 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8b4ff-9c73-4ed2-b5fb-7948297c14b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:53.346240 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8b4ff-9c73-4ed2-b5fb-7948297c14b8 admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:53.346970 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b8b4ff-9c73-4ed2-b5fb-7948297c14b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:53.348931 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b8b4ff-9c73-4ed2-b5fb-7948297c14b8 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:53.364855 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79532f31-bf62-4c95-866f-c20243b90e87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc655b1bf30b41aaa9c2de2b2db0cdd2, user_id=e6047b7680b745a1a8972fe7fdb4a1f6, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:53.368541 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79532f31-bf62-4c95-866f-c20243b90e87 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:53.384453 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1271/2551] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:55:53 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:55:53.388933 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8b4ff-9c73-4ed2-b5fb-7948297c14b8 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:53.409341 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1281/2552] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:55:53 2022] PUT /identity/v3/projects/dc655b1bf30b41aaa9c2de2b2db0cdd2/users/e6047b7680b745a1a8972fe7fdb4a1f6/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:55:53.411515 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8b4ff-9c73-4ed2-b5fb-7948297c14b8 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:53.421324 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6f012-c2c4-4329-8de9-eb3957c8daea None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:53.421773 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6f012-c2c4-4329-8de9-eb3957c8daea None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:53.422131 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6f012-c2c4-4329-8de9-eb3957c8daea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:53.437044 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:55:53.458964 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-a2e6f012-c2c4-4329-8de9-eb3957c8daea None None] MFA Rules not processed for user `e6047b7680b745a1a8972fe7fdb4a1f6`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:55:53.460406 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-a2e6f012-c2c4-4329-8de9-eb3957c8daea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:53.595069 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6970a867-568d-4999-b967-54b670853db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:53.595722 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970a867-568d-4999-b967-54b670853db0 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:53.596004 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970a867-568d-4999-b967-54b670853db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:53.596289 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970a867-568d-4999-b967-54b670853db0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:53.597080 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6970a867-568d-4999-b967-54b670853db0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:53.601771 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1282/2553] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:55:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:55:53.612256 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6970a867-568d-4999-b967-54b670853db0 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:53.617469 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6f012-c2c4-4329-8de9-eb3957c8daea None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:53.619714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-a2e6f012-c2c4-4329-8de9-eb3957c8daea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:53.638400 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1272/2554] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:55:53.642524 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6970a867-568d-4999-b967-54b670853db0 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:53.655501 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6970a867-568d-4999-b967-54b670853db0 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:53.661869 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a16b72-93f7-47e0-b8d0-5ce69b4117ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:53.662836 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a16b72-93f7-47e0-b8d0-5ce69b4117ed admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:53.663305 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a16b72-93f7-47e0-b8d0-5ce69b4117ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:53.663707 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a16b72-93f7-47e0-b8d0-5ce69b4117ed admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:53.664581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a16b72-93f7-47e0-b8d0-5ce69b4117ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:53.666505 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a16b72-93f7-47e0-b8d0-5ce69b4117ed admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:53.683095 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1273/2555] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:55:53.692897 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a16b72-93f7-47e0-b8d0-5ce69b4117ed admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:53.707682 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a16b72-93f7-47e0-b8d0-5ce69b4117ed admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:53.713362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36653b15-4a4b-4146-9ea6-7e6112c226b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:53.714684 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36653b15-4a4b-4146-9ea6-7e6112c226b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:53.715131 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36653b15-4a4b-4146-9ea6-7e6112c226b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:53.716444 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36653b15-4a4b-4146-9ea6-7e6112c226b1 admin admin] PATH_INFO: `/v3/projects/a3c446b1e01346f89bd4b1cb3b80f876/users/aef37fab20514f9999b8bbafde3f6441/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:53.724869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6f012-c2c4-4329-8de9-eb3957c8daea None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:53.731939 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e194527-d25b-4a18-9f39-a29bbddecd9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:53.732652 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e194527-d25b-4a18-9f39-a29bbddecd9d service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:53.732852 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e194527-d25b-4a18-9f39-a29bbddecd9d service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:53.733196 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e194527-d25b-4a18-9f39-a29bbddecd9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:53.736311 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36653b15-4a4b-4146-9ea6-7e6112c226b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3c446b1e01346f89bd4b1cb3b80f876, user_id=aef37fab20514f9999b8bbafde3f6441, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:53.739257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36653b15-4a4b-4146-9ea6-7e6112c226b1 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:53.744798 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e194527-d25b-4a18-9f39-a29bbddecd9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:53.747609 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e194527-d25b-4a18-9f39-a29bbddecd9d service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:53.757881 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1283/2556] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:55:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 13:55:53.778813 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1274/2557] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:55:53 2022] PUT /identity/v3/projects/a3c446b1e01346f89bd4b1cb3b80f876/users/aef37fab20514f9999b8bbafde3f6441/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:55:53.780604 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e194527-d25b-4a18-9f39-a29bbddecd9d service cinder] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:53.786178 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b68501-fedf-4a02-8ad9-2e66df4538e6 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:53.786980 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b68501-fedf-4a02-8ad9-2e66df4538e6 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:53.787660 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b68501-fedf-4a02-8ad9-2e66df4538e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:53.819278 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e194527-d25b-4a18-9f39-a29bbddecd9d service cinder] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:53.829216 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-65b68501-fedf-4a02-8ad9-2e66df4538e6 None None] MFA Rules not processed for user `aef37fab20514f9999b8bbafde3f6441`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:55:53.830592 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-65b68501-fedf-4a02-8ad9-2e66df4538e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:53.905442 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252fec9-efa0-41d4-a164-85fc2d9104ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:53.906873 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252fec9-efa0-41d4-a164-85fc2d9104ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:53.907342 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252fec9-efa0-41d4-a164-85fc2d9104ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:53.907817 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252fec9-efa0-41d4-a164-85fc2d9104ea admin admin] PATH_INFO: `/v3/users/e6047b7680b745a1a8972fe7fdb4a1f6` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:53.909811 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d252fec9-efa0-41d4-a164-85fc2d9104ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6047b7680b745a1a8972fe7fdb4a1f6)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:53.912091 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d252fec9-efa0-41d4-a164-85fc2d9104ea admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:53.937068 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1275/2558] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:55:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:55:53.939066 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b68501-fedf-4a02-8ad9-2e66df4538e6 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:53.946250 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:55:53.951726 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b68501-fedf-4a02-8ad9-2e66df4538e6 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:53.958326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e72d74-c6f0-4c47-b2b7-b3c9bca97268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:53.959011 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e72d74-c6f0-4c47-b2b7-b3c9bca97268 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:53.959312 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e72d74-c6f0-4c47-b2b7-b3c9bca97268 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:53.959587 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e72d74-c6f0-4c47-b2b7-b3c9bca97268 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:53.960374 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e72d74-c6f0-4c47-b2b7-b3c9bca97268 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:53.962641 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e72d74-c6f0-4c47-b2b7-b3c9bca97268 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:53.990155 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:55:53.999870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-d252fec9-efa0-41d4-a164-85fc2d9104ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6047b7680b745a1a8972fe7fdb4a1f6'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:54.016355 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-d252fec9-efa0-41d4-a164-85fc2d9104ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6047b7680b745a1a8972fe7fdb4a1f6'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:54.017690 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1276/2559] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:55:53 2022] POST /identity/v3/projects => generated 393 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:55:54.021755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e72d74-c6f0-4c47-b2b7-b3c9bca97268 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:54.029776 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-d252fec9-efa0-41d4-a164-85fc2d9104ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6047b7680b745a1a8972fe7fdb4a1f6'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:54.030298 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-d252fec9-efa0-41d4-a164-85fc2d9104ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6047b7680b745a1a8972fe7fdb4a1f6'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:54.048195 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-d252fec9-efa0-41d4-a164-85fc2d9104ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6047b7680b745a1a8972fe7fdb4a1f6'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:54.055748 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1284/2560] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:55:53 2022] DELETE /identity/v3/users/e6047b7680b745a1a8972fe7fdb4a1f6 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:55:54.061937 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25ad45-9a1d-435e-a74a-a7a80f790805 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:54.062316 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25ad45-9a1d-435e-a74a-a7a80f790805 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:54.063075 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25ad45-9a1d-435e-a74a-a7a80f790805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:54.086464 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-5a25ad45-9a1d-435e-a74a-a7a80f790805 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:55:54.087681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-5a25ad45-9a1d-435e-a74a-a7a80f790805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:54.105218 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e72d74-c6f0-4c47-b2b7-b3c9bca97268 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:54.168949 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b7047-a14e-43aa-9c74-e4b1f709f698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:54.170173 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b7047-a14e-43aa-9c74-e4b1f709f698 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:54.170801 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b7047-a14e-43aa-9c74-e4b1f709f698 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:54.171291 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b7047-a14e-43aa-9c74-e4b1f709f698 admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:54.172208 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934b7047-a14e-43aa-9c74-e4b1f709f698 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:54.174750 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934b7047-a14e-43aa-9c74-e4b1f709f698 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:54.196237 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1285/2561] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:55:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:55:54.206485 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1277/2562] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:55:54 2022] POST /identity/v3/users => generated 303 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:55:54.206767 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a25ad45-9a1d-435e-a74a-a7a80f790805 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:54.208008 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b7047-a14e-43aa-9c74-e4b1f709f698 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:54.208322 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-5a25ad45-9a1d-435e-a74a-a7a80f790805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:54.210278 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-934b7047-a14e-43aa-9c74-e4b1f709f698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:54.225315 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b7047-a14e-43aa-9c74-e4b1f709f698 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:54.230167 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a9629-74bc-4030-b0ee-5afa6a50fa46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:54.230833 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a9629-74bc-4030-b0ee-5afa6a50fa46 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:54.231229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a9629-74bc-4030-b0ee-5afa6a50fa46 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:54.232652 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a9629-74bc-4030-b0ee-5afa6a50fa46 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:54.233463 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a9629-74bc-4030-b0ee-5afa6a50fa46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:54.245211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a9629-74bc-4030-b0ee-5afa6a50fa46 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:54.249122 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1278/2563] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:55:54.253273 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a9629-74bc-4030-b0ee-5afa6a50fa46 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:54.263683 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a9629-74bc-4030-b0ee-5afa6a50fa46 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:54.268816 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9679453-1730-4127-9eeb-51a2162d2298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:54.269738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9679453-1730-4127-9eeb-51a2162d2298 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:54.270023 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9679453-1730-4127-9eeb-51a2162d2298 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:54.270317 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9679453-1730-4127-9eeb-51a2162d2298 admin admin] PATH_INFO: `/v3/projects/3a731723872145b98025d66412b10475/users/b67439cf47124692bc2869c9be7ae8e0/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:54.287496 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9679453-1730-4127-9eeb-51a2162d2298 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a731723872145b98025d66412b10475, user_id=b67439cf47124692bc2869c9be7ae8e0, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:54.290544 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9679453-1730-4127-9eeb-51a2162d2298 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:54.311909 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1279/2564] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:55:54 2022] PUT /identity/v3/projects/3a731723872145b98025d66412b10475/users/b67439cf47124692bc2869c9be7ae8e0/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:55:54.316125 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9679453-1730-4127-9eeb-51a2162d2298 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:54.327320 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a25ad45-9a1d-435e-a74a-a7a80f790805 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:54.381202 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:55:54.399943 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78d26d-5c75-4545-8be0-c901d11c40b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:54.400604 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78d26d-5c75-4545-8be0-c901d11c40b5 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:54.400884 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78d26d-5c75-4545-8be0-c901d11c40b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:54.401186 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78d26d-5c75-4545-8be0-c901d11c40b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:54.412660 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea78d26d-5c75-4545-8be0-c901d11c40b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:54.414828 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea78d26d-5c75-4545-8be0-c901d11c40b5 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:54.427050 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1286/2565] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:55:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 13:55:54.429862 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9679453-1730-4127-9eeb-51a2162d2298 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:54.435326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c65a12-2086-4694-bcbc-6b8e0c1688a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:54.436010 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c65a12-2086-4694-bcbc-6b8e0c1688a6 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:54.436421 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c65a12-2086-4694-bcbc-6b8e0c1688a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:54.436716 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c65a12-2086-4694-bcbc-6b8e0c1688a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:54.437493 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c65a12-2086-4694-bcbc-6b8e0c1688a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:54.439105 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c65a12-2086-4694-bcbc-6b8e0c1688a6 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:54.451550 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1280/2566] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:55:54.455679 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78d26d-5c75-4545-8be0-c901d11c40b5 service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:54.466193 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78d26d-5c75-4545-8be0-c901d11c40b5 service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:54.471135 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8087e9-7b3a-490c-9e76-4b33c78d18b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:54.472117 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8087e9-7b3a-490c-9e76-4b33c78d18b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:54.472301 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8087e9-7b3a-490c-9e76-4b33c78d18b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:54.472578 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8087e9-7b3a-490c-9e76-4b33c78d18b0 admin admin] PATH_INFO: `/v3/projects/3a731723872145b98025d66412b10475/users/b67439cf47124692bc2869c9be7ae8e0/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:54.474859 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8087e9-7b3a-490c-9e76-4b33c78d18b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a731723872145b98025d66412b10475, user_id=b67439cf47124692bc2869c9be7ae8e0, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:54.477520 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8087e9-7b3a-490c-9e76-4b33c78d18b0 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:54.490906 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1287/2567] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:55:54 2022] PUT /identity/v3/projects/3a731723872145b98025d66412b10475/users/b67439cf47124692bc2869c9be7ae8e0/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:55:54.502484 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4f9e0-0099-413b-ba86-7212d1faa42e None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:54.502872 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4f9e0-0099-413b-ba86-7212d1faa42e None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:54.503215 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4f9e0-0099-413b-ba86-7212d1faa42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:54.541253 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-d7e4f9e0-0099-413b-ba86-7212d1faa42e None None] MFA Rules not processed for user `b67439cf47124692bc2869c9be7ae8e0`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:55:54.542992 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-d7e4f9e0-0099-413b-ba86-7212d1faa42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:54.631857 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1281/2568] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:55:54 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:55:54.643366 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8087e9-7b3a-490c-9e76-4b33c78d18b0 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:54.645262 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-5a8087e9-7b3a-490c-9e76-4b33c78d18b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:54.652225 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:55:54.742336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8087e9-7b3a-490c-9e76-4b33c78d18b0 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:54.751032 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229aa53-e520-4caa-9417-dd0af19f9e14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:54.751809 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229aa53-e520-4caa-9417-dd0af19f9e14 service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:54.752164 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229aa53-e520-4caa-9417-dd0af19f9e14 service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:54.752514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229aa53-e520-4caa-9417-dd0af19f9e14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:54.764451 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4229aa53-e520-4caa-9417-dd0af19f9e14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:54.766570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4229aa53-e520-4caa-9417-dd0af19f9e14 service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:54.779511 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1288/2569] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:55:54 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:55:54.786223 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4f9e0-0099-413b-ba86-7212d1faa42e None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:54.788722 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-d7e4f9e0-0099-413b-ba86-7212d1faa42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:54.834898 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229aa53-e520-4caa-9417-dd0af19f9e14 service cinder] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:54.836334 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-4229aa53-e520-4caa-9417-dd0af19f9e14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:54.847645 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229aa53-e520-4caa-9417-dd0af19f9e14 service cinder] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:54.851282 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a734f306-a003-4acb-8398-b0658f426b41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:54.851917 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734f306-a003-4acb-8398-b0658f426b41 service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:54.852210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734f306-a003-4acb-8398-b0658f426b41 service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:54.852491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734f306-a003-4acb-8398-b0658f426b41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:54.854323 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-a734f306-a003-4acb-8398-b0658f426b41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:54.874682 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4f9e0-0099-413b-ba86-7212d1faa42e None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:54.879324 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aac53b-bc58-4c78-8d5b-e43c5475e230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:54.880176 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aac53b-bc58-4c78-8d5b-e43c5475e230 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:54.880465 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aac53b-bc58-4c78-8d5b-e43c5475e230 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:54.880740 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aac53b-bc58-4c78-8d5b-e43c5475e230 admin admin] PATH_INFO: `/v3/projects/dc655b1bf30b41aaa9c2de2b2db0cdd2` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:54.881995 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aac53b-bc58-4c78-8d5b-e43c5475e230 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc655b1bf30b41aaa9c2de2b2db0cdd2)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:54.883873 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aac53b-bc58-4c78-8d5b-e43c5475e230 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:54.915765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a734f306-a003-4acb-8398-b0658f426b41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:54.917345 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a734f306-a003-4acb-8398-b0658f426b41 service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:54.927132 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1289/2570] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:55:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 13:55:54.934930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-97aac53b-bc58-4c78-8d5b-e43c5475e230 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc655b1bf30b41aaa9c2de2b2db0cdd2', 'request_id': 'req-97aac53b-bc58-4c78-8d5b-e43c5475e230'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:54.941159 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-97aac53b-bc58-4c78-8d5b-e43c5475e230 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc655b1bf30b41aaa9c2de2b2db0cdd2', 'request_id': 'req-97aac53b-bc58-4c78-8d5b-e43c5475e230'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:54.948747 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-97aac53b-bc58-4c78-8d5b-e43c5475e230 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc655b1bf30b41aaa9c2de2b2db0cdd2', 'request_id': 'req-97aac53b-bc58-4c78-8d5b-e43c5475e230'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:54.956084 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:55:54.964494 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-97aac53b-bc58-4c78-8d5b-e43c5475e230 admin admin] The token cache is being invalidate because project dc655b1bf30b41aaa9c2de2b2db0cdd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:55:54.965324 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-97aac53b-bc58-4c78-8d5b-e43c5475e230 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:54.966376 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-97aac53b-bc58-4c78-8d5b-e43c5475e230 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:54.967989 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1282/2571] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:55:54 2022] DELETE /identity/v3/projects/dc655b1bf30b41aaa9c2de2b2db0cdd2 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:55:54.973125 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2a83f-5924-4caf-8348-02b1fc5b11ad None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:54.973484 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2a83f-5924-4caf-8348-02b1fc5b11ad None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:54.973659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2a83f-5924-4caf-8348-02b1fc5b11ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:54.991396 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-5ab2a83f-5924-4caf-8348-02b1fc5b11ad None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:55:54.992588 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-5ab2a83f-5924-4caf-8348-02b1fc5b11ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:55.057961 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1290/2572] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:55:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:55:55.067034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893de0fa-5943-485f-af77-8646dfee8641 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:55.067806 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893de0fa-5943-485f-af77-8646dfee8641 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:55.068454 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893de0fa-5943-485f-af77-8646dfee8641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:55.093737 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-893de0fa-5943-485f-af77-8646dfee8641 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:55:55.094944 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-893de0fa-5943-485f-af77-8646dfee8641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:55.103787 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1283/2573] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:55:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:55:55.108623 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2a83f-5924-4caf-8348-02b1fc5b11ad None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:55.118521 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2a83f-5924-4caf-8348-02b1fc5b11ad None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:55.124185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf531f6-61fe-495d-a2c6-50aa7bf2876a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:55.124838 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf531f6-61fe-495d-a2c6-50aa7bf2876a admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:55.125162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf531f6-61fe-495d-a2c6-50aa7bf2876a admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:55.125436 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf531f6-61fe-495d-a2c6-50aa7bf2876a admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:55.126159 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf531f6-61fe-495d-a2c6-50aa7bf2876a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:55:55.126460 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf531f6-61fe-495d-a2c6-50aa7bf2876a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:55.128028 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf531f6-61fe-495d-a2c6-50aa7bf2876a admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:55.145285 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1291/2574] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:55:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:55:55.146354 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893de0fa-5943-485f-af77-8646dfee8641 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:55.156118 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893de0fa-5943-485f-af77-8646dfee8641 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:55.160916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e696d90-36af-4b16-8c2c-3fe589efa7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:55.161613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e696d90-36af-4b16-8c2c-3fe589efa7be admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:55.161898 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e696d90-36af-4b16-8c2c-3fe589efa7be admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:55.162175 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e696d90-36af-4b16-8c2c-3fe589efa7be admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:55.162868 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e696d90-36af-4b16-8c2c-3fe589efa7be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:55.164760 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e696d90-36af-4b16-8c2c-3fe589efa7be admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:55.196775 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1284/2575] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:55:55 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:55:55.201672 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf531f6-61fe-495d-a2c6-50aa7bf2876a admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:55.216128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf531f6-61fe-495d-a2c6-50aa7bf2876a admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:55.252696 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=100 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:55:55.281337 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db6cac-7bf5-4caa-bcee-82a7a46d6d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:55.282005 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db6cac-7bf5-4caa-bcee-82a7a46d6d15 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:55.282313 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db6cac-7bf5-4caa-bcee-82a7a46d6d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:55.282588 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db6cac-7bf5-4caa-bcee-82a7a46d6d15 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:55.283373 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db6cac-7bf5-4caa-bcee-82a7a46d6d15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:55.285938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db6cac-7bf5-4caa-bcee-82a7a46d6d15 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:55.312953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1292/2576] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:55:55 2022] POST /identity/v3/users => generated 288 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:55:55.317187 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e696d90-36af-4b16-8c2c-3fe589efa7be admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:55.327463 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e696d90-36af-4b16-8c2c-3fe589efa7be admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:55.332045 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc1f48c-0b28-4da3-bc21-8ddb402f4a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:55.332695 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1f48c-0b28-4da3-bc21-8ddb402f4a16 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:55.332977 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1f48c-0b28-4da3-bc21-8ddb402f4a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:55.333362 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1f48c-0b28-4da3-bc21-8ddb402f4a16 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:55.335592 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc1f48c-0b28-4da3-bc21-8ddb402f4a16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:55.337301 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc1f48c-0b28-4da3-bc21-8ddb402f4a16 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:55.347882 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1285/2577] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:55:55.351811 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db6cac-7bf5-4caa-bcee-82a7a46d6d15 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:55.362202 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db6cac-7bf5-4caa-bcee-82a7a46d6d15 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:55.366714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e27185-e53f-494d-920d-50c7df59a277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:55.367365 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e27185-e53f-494d-920d-50c7df59a277 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:55.367643 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e27185-e53f-494d-920d-50c7df59a277 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:55.367922 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e27185-e53f-494d-920d-50c7df59a277 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:55.368640 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e27185-e53f-494d-920d-50c7df59a277 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:55.370255 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e27185-e53f-494d-920d-50c7df59a277 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:55.380611 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1293/2578] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:55:55.384607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc1f48c-0b28-4da3-bc21-8ddb402f4a16 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:55.394526 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc1f48c-0b28-4da3-bc21-8ddb402f4a16 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:55.399129 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac2718-be37-46ea-abcc-9e94cb03c051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:55.399930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac2718-be37-46ea-abcc-9e94cb03c051 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:55.400211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac2718-be37-46ea-abcc-9e94cb03c051 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:55.400483 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac2718-be37-46ea-abcc-9e94cb03c051 admin admin] PATH_INFO: `/v3/projects/093c882a86024b269c1f90973f9ddcf1/users/a75bdacc0fe5456cac89f8699bcf8f54/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:55.414482 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac2718-be37-46ea-abcc-9e94cb03c051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=093c882a86024b269c1f90973f9ddcf1, user_id=a75bdacc0fe5456cac89f8699bcf8f54, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:55.417134 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac2718-be37-46ea-abcc-9e94cb03c051 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:55.430699 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1286/2579] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:55:55 2022] PUT /identity/v3/projects/093c882a86024b269c1f90973f9ddcf1/users/a75bdacc0fe5456cac89f8699bcf8f54/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:55:55.441320 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca380e-3179-4526-95ac-df12a85a1ddf None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:55.441608 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca380e-3179-4526-95ac-df12a85a1ddf None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:55.441884 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca380e-3179-4526-95ac-df12a85a1ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:55.469638 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-4fca380e-3179-4526-95ac-df12a85a1ddf None None] MFA Rules not processed for user `a75bdacc0fe5456cac89f8699bcf8f54`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:55:55.470836 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-4fca380e-3179-4526-95ac-df12a85a1ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:55.533857 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1294/2580] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:55:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:55:55.545983 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac2718-be37-46ea-abcc-9e94cb03c051 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:55.547647 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-eeac2718-be37-46ea-abcc-9e94cb03c051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:55.603148 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac2718-be37-46ea-abcc-9e94cb03c051 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:55.607361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9656d7-cbc1-4b98-875c-179e1e18dd3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:55.607985 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9656d7-cbc1-4b98-875c-179e1e18dd3e service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:55.608272 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9656d7-cbc1-4b98-875c-179e1e18dd3e service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:55.608546 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9656d7-cbc1-4b98-875c-179e1e18dd3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:55.616640 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9656d7-cbc1-4b98-875c-179e1e18dd3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:55.618083 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9656d7-cbc1-4b98-875c-179e1e18dd3e service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:55.627208 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1287/2581] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:55:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:55:58.211054 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca380e-3179-4526-95ac-df12a85a1ddf None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:58.212539 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-4fca380e-3179-4526-95ac-df12a85a1ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:58.285608 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca380e-3179-4526-95ac-df12a85a1ddf None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:58.290238 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a8deef-c0ff-40c6-bb00-817fb1be0805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:58.291236 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8deef-c0ff-40c6-bb00-817fb1be0805 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:58.291612 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8deef-c0ff-40c6-bb00-817fb1be0805 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:58.291990 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8deef-c0ff-40c6-bb00-817fb1be0805 admin admin] PATH_INFO: `/v3/users/aef37fab20514f9999b8bbafde3f6441` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:58.293314 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a8deef-c0ff-40c6-bb00-817fb1be0805 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aef37fab20514f9999b8bbafde3f6441)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:58.295533 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a8deef-c0ff-40c6-bb00-817fb1be0805 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:58.343896 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:55:58.346268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b4a8deef-c0ff-40c6-bb00-817fb1be0805 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aef37fab20514f9999b8bbafde3f6441'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:58.358750 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b4a8deef-c0ff-40c6-bb00-817fb1be0805 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aef37fab20514f9999b8bbafde3f6441'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:58.366056 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b4a8deef-c0ff-40c6-bb00-817fb1be0805 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aef37fab20514f9999b8bbafde3f6441'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:58.366670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b4a8deef-c0ff-40c6-bb00-817fb1be0805 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aef37fab20514f9999b8bbafde3f6441'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:58.378184 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b4a8deef-c0ff-40c6-bb00-817fb1be0805 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aef37fab20514f9999b8bbafde3f6441'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:58.383928 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1295/2582] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:55:58 2022] DELETE /identity/v3/users/aef37fab20514f9999b8bbafde3f6441 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:55:58.388956 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6ea16-7a09-45f7-8a73-b7a548018bb6 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:58.389451 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6ea16-7a09-45f7-8a73-b7a548018bb6 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:58.389572 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6ea16-7a09-45f7-8a73-b7a548018bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:58.406977 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-e8b6ea16-7a09-45f7-8a73-b7a548018bb6 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:55:58.408151 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-e8b6ea16-7a09-45f7-8a73-b7a548018bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:58.465530 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1288/2583] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:55:58.475099 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a8deef-c0ff-40c6-bb00-817fb1be0805 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:58.476547 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-b4a8deef-c0ff-40c6-bb00-817fb1be0805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:58.533324 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a8deef-c0ff-40c6-bb00-817fb1be0805 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:58.536916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3416e2-e65e-4414-9e1c-3fbf346ec5a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:58.537654 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3416e2-e65e-4414-9e1c-3fbf346ec5a9 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:58.537941 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3416e2-e65e-4414-9e1c-3fbf346ec5a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:58.538221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3416e2-e65e-4414-9e1c-3fbf346ec5a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:58.547789 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3416e2-e65e-4414-9e1c-3fbf346ec5a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:58.549274 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3416e2-e65e-4414-9e1c-3fbf346ec5a9 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:58.558928 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1296/2584] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:55:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:55:58.820736 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b6ea16-7a09-45f7-8a73-b7a548018bb6 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:58.822243 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-e8b6ea16-7a09-45f7-8a73-b7a548018bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:58.835677 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b6ea16-7a09-45f7-8a73-b7a548018bb6 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:58.840232 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a916ee63-31c6-4b57-bfa5-ba35b54d9eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:58.841940 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916ee63-31c6-4b57-bfa5-ba35b54d9eb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:58.842705 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916ee63-31c6-4b57-bfa5-ba35b54d9eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:58.842994 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916ee63-31c6-4b57-bfa5-ba35b54d9eb8 admin admin] PATH_INFO: `/v3/projects/a3c446b1e01346f89bd4b1cb3b80f876` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:58.845461 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a916ee63-31c6-4b57-bfa5-ba35b54d9eb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3c446b1e01346f89bd4b1cb3b80f876)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:58.848598 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a916ee63-31c6-4b57-bfa5-ba35b54d9eb8 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:58.882064 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:55:58.897509 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a916ee63-31c6-4b57-bfa5-ba35b54d9eb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3c446b1e01346f89bd4b1cb3b80f876', 'request_id': 'req-a916ee63-31c6-4b57-bfa5-ba35b54d9eb8'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:58.903902 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a916ee63-31c6-4b57-bfa5-ba35b54d9eb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3c446b1e01346f89bd4b1cb3b80f876', 'request_id': 'req-a916ee63-31c6-4b57-bfa5-ba35b54d9eb8'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:58.911675 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a916ee63-31c6-4b57-bfa5-ba35b54d9eb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3c446b1e01346f89bd4b1cb3b80f876', 'request_id': 'req-a916ee63-31c6-4b57-bfa5-ba35b54d9eb8'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:58.922616 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a916ee63-31c6-4b57-bfa5-ba35b54d9eb8 admin admin] The token cache is being invalidate because project a3c446b1e01346f89bd4b1cb3b80f876 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:55:58.922902 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a916ee63-31c6-4b57-bfa5-ba35b54d9eb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:58.923519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a916ee63-31c6-4b57-bfa5-ba35b54d9eb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:58.925329 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1289/2585] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:55:58 2022] DELETE /identity/v3/projects/a3c446b1e01346f89bd4b1cb3b80f876 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:55:58.929250 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3416e2-e65e-4414-9e1c-3fbf346ec5a9 service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:58.939556 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3416e2-e65e-4414-9e1c-3fbf346ec5a9 service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:58.994520 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a368f-0da7-4483-a771-3b91db82c25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:58.995384 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a368f-0da7-4483-a771-3b91db82c25d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:58.995699 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a368f-0da7-4483-a771-3b91db82c25d admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:58.995980 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a368f-0da7-4483-a771-3b91db82c25d admin admin] PATH_INFO: `/v3/users/b67439cf47124692bc2869c9be7ae8e0` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:58.997250 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a368f-0da7-4483-a771-3b91db82c25d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b67439cf47124692bc2869c9be7ae8e0)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:58.999158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a368f-0da7-4483-a771-3b91db82c25d admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:59.044678 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-695a368f-0da7-4483-a771-3b91db82c25d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b67439cf47124692bc2869c9be7ae8e0'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:59.052284 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-695a368f-0da7-4483-a771-3b91db82c25d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b67439cf47124692bc2869c9be7ae8e0'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:59.059740 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-695a368f-0da7-4483-a771-3b91db82c25d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b67439cf47124692bc2869c9be7ae8e0'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:59.060637 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-695a368f-0da7-4483-a771-3b91db82c25d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b67439cf47124692bc2869c9be7ae8e0'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:59.062641 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:55:59.076635 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-695a368f-0da7-4483-a771-3b91db82c25d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b67439cf47124692bc2869c9be7ae8e0'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:59.083069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1297/2586] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:55:58 2022] DELETE /identity/v3/users/b67439cf47124692bc2869c9be7ae8e0 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:55:59.406571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a916ee63-31c6-4b57-bfa5-ba35b54d9eb8 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:59.409453 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a916ee63-31c6-4b57-bfa5-ba35b54d9eb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:59.425584 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a916ee63-31c6-4b57-bfa5-ba35b54d9eb8 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:59.492673 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d90b1-68e4-4523-aea6-b9e1328430d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:59.493619 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d90b1-68e4-4523-aea6-b9e1328430d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:59.494512 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d90b1-68e4-4523-aea6-b9e1328430d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:59.494512 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d90b1-68e4-4523-aea6-b9e1328430d5 admin admin] PATH_INFO: `/v3/projects/3a731723872145b98025d66412b10475` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:59.495487 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d90b1-68e4-4523-aea6-b9e1328430d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a731723872145b98025d66412b10475)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:59.498174 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d90b1-68e4-4523-aea6-b9e1328430d5 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:59.547903 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-6c7d90b1-68e4-4523-aea6-b9e1328430d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a731723872145b98025d66412b10475', 'request_id': 'req-6c7d90b1-68e4-4523-aea6-b9e1328430d5'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:59.554638 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-6c7d90b1-68e4-4523-aea6-b9e1328430d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a731723872145b98025d66412b10475', 'request_id': 'req-6c7d90b1-68e4-4523-aea6-b9e1328430d5'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:59.563119 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-6c7d90b1-68e4-4523-aea6-b9e1328430d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a731723872145b98025d66412b10475', 'request_id': 'req-6c7d90b1-68e4-4523-aea6-b9e1328430d5'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:59.574055 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-6c7d90b1-68e4-4523-aea6-b9e1328430d5 admin admin] The token cache is being invalidate because project 3a731723872145b98025d66412b10475 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:55:59.574371 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-6c7d90b1-68e4-4523-aea6-b9e1328430d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:59.575000 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-6c7d90b1-68e4-4523-aea6-b9e1328430d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:55:59.576741 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1290/2587] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:55:59 2022] DELETE /identity/v3/projects/3a731723872145b98025d66412b10475 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:55:59.582379 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81e070-74e9-4010-8103-10b907cc4515 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:59.582869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81e070-74e9-4010-8103-10b907cc4515 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:59.582977 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81e070-74e9-4010-8103-10b907cc4515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:59.604086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-5f81e070-74e9-4010-8103-10b907cc4515 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:55:59.605346 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-5f81e070-74e9-4010-8103-10b907cc4515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:59.665098 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1298/2588] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:55:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:55:59.674427 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a79515-a68f-42a5-b033-d9dd746fb2b1 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:59.674714 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a79515-a68f-42a5-b033-d9dd746fb2b1 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:59.675246 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a79515-a68f-42a5-b033-d9dd746fb2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:59.693925 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-d2a79515-a68f-42a5-b033-d9dd746fb2b1 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:55:59.695040 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-d2a79515-a68f-42a5-b033-d9dd746fb2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:55:59.703117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1291/2589] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:55:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:55:59.707346 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81e070-74e9-4010-8103-10b907cc4515 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:59.716887 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81e070-74e9-4010-8103-10b907cc4515 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:59.721485 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803df16-c1b7-40e6-995d-5e8086f84357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:59.722128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803df16-c1b7-40e6-995d-5e8086f84357 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:59.722408 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803df16-c1b7-40e6-995d-5e8086f84357 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:59.722681 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803df16-c1b7-40e6-995d-5e8086f84357 admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:59.723400 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a803df16-c1b7-40e6-995d-5e8086f84357 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:55:59.723702 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a803df16-c1b7-40e6-995d-5e8086f84357 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:59.725297 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a803df16-c1b7-40e6-995d-5e8086f84357 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:59.737245 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1299/2590] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:55:59.741307 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a79515-a68f-42a5-b033-d9dd746fb2b1 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:59.751230 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a79515-a68f-42a5-b033-d9dd746fb2b1 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:59.755849 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0890328-fdf9-4da9-8531-98297ae63845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:59.756500 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0890328-fdf9-4da9-8531-98297ae63845 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:59.756786 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0890328-fdf9-4da9-8531-98297ae63845 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:59.757095 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0890328-fdf9-4da9-8531-98297ae63845 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:59.757794 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0890328-fdf9-4da9-8531-98297ae63845 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:59.759692 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0890328-fdf9-4da9-8531-98297ae63845 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:59.784536 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1292/2591] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:55:59 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:55:59.788387 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803df16-c1b7-40e6-995d-5e8086f84357 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:59.798638 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803df16-c1b7-40e6-995d-5e8086f84357 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:59.854613 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b56184-dda2-4536-9649-bc3336de68e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:59.855279 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b56184-dda2-4536-9649-bc3336de68e2 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:59.855566 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b56184-dda2-4536-9649-bc3336de68e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:59.855848 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b56184-dda2-4536-9649-bc3336de68e2 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:59.857199 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b56184-dda2-4536-9649-bc3336de68e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:59.859116 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b56184-dda2-4536-9649-bc3336de68e2 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:59.879757 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1300/2592] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:55:59 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:55:59.883192 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0890328-fdf9-4da9-8531-98297ae63845 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:59.893757 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0890328-fdf9-4da9-8531-98297ae63845 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:59.898364 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a32cba7-06a2-4d23-9e0e-57a6ee390e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:59.899046 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32cba7-06a2-4d23-9e0e-57a6ee390e4b admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:59.899349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32cba7-06a2-4d23-9e0e-57a6ee390e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:59.899630 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32cba7-06a2-4d23-9e0e-57a6ee390e4b admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:59.900356 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a32cba7-06a2-4d23-9e0e-57a6ee390e4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:59.901982 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a32cba7-06a2-4d23-9e0e-57a6ee390e4b admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:59.913739 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1293/2593] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:55:59.917586 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b56184-dda2-4536-9649-bc3336de68e2 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:59.928233 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b56184-dda2-4536-9649-bc3336de68e2 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:59.932719 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d072c-d2e9-4283-b763-19d6a901df2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:59.933414 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d072c-d2e9-4283-b763-19d6a901df2e admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:59.933693 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d072c-d2e9-4283-b763-19d6a901df2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:59.933968 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d072c-d2e9-4283-b763-19d6a901df2e admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:59.935385 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7d072c-d2e9-4283-b763-19d6a901df2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:59.936950 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7d072c-d2e9-4283-b763-19d6a901df2e admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:55:59.948436 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1301/2594] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:55:59.952491 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a32cba7-06a2-4d23-9e0e-57a6ee390e4b admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:55:59.962460 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a32cba7-06a2-4d23-9e0e-57a6ee390e4b admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:55:59.967666 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b27cc-9c25-4bf8-810c-67bbd110b0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:55:59.968538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b27cc-9c25-4bf8-810c-67bbd110b0dc admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:55:59.968824 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b27cc-9c25-4bf8-810c-67bbd110b0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:55:59.969138 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b27cc-9c25-4bf8-810c-67bbd110b0dc admin admin] PATH_INFO: `/v3/projects/ec0ddb31fecf45d29fad8dfd5efd6c23/users/b889778476eb4b4eb316e16bf20415fe/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:55:59.972127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:55:59.985754 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b27cc-9c25-4bf8-810c-67bbd110b0dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec0ddb31fecf45d29fad8dfd5efd6c23, user_id=b889778476eb4b4eb316e16bf20415fe, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:55:59.988446 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b27cc-9c25-4bf8-810c-67bbd110b0dc admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:00.000601 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1294/2595] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:55:59 2022] PUT /identity/v3/projects/ec0ddb31fecf45d29fad8dfd5efd6c23/users/b889778476eb4b4eb316e16bf20415fe/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:56:00.010962 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bba229-9370-4104-83d0-c65a6876b3d6 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:00.011252 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bba229-9370-4104-83d0-c65a6876b3d6 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:00.011527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bba229-9370-4104-83d0-c65a6876b3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:00.038252 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-c8bba229-9370-4104-83d0-c65a6876b3d6 None None] MFA Rules not processed for user `b889778476eb4b4eb316e16bf20415fe`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:56:00.039457 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-c8bba229-9370-4104-83d0-c65a6876b3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:00.076325 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:56:00.102982 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1302/2596] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:56:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:56:00.107634 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b27cc-9c25-4bf8-810c-67bbd110b0dc admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:00.167633 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b27cc-9c25-4bf8-810c-67bbd110b0dc admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:00.172392 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f451a5e-1e11-46e6-a3f6-5fa2950c3039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:00.173068 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f451a5e-1e11-46e6-a3f6-5fa2950c3039 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:00.173359 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f451a5e-1e11-46e6-a3f6-5fa2950c3039 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:00.173641 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f451a5e-1e11-46e6-a3f6-5fa2950c3039 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:00.174323 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f451a5e-1e11-46e6-a3f6-5fa2950c3039 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:00.176477 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f451a5e-1e11-46e6-a3f6-5fa2950c3039 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:00.201668 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1295/2597] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:56:00 2022] POST /identity/v3/projects => generated 383 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:56:00.205667 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bba229-9370-4104-83d0-c65a6876b3d6 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:00.216757 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bba229-9370-4104-83d0-c65a6876b3d6 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:00.274925 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b8627-c929-4ac9-8c39-eb10abca50e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:00.275753 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b8627-c929-4ac9-8c39-eb10abca50e3 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:00.276100 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b8627-c929-4ac9-8c39-eb10abca50e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:00.276381 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b8627-c929-4ac9-8c39-eb10abca50e3 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:00.277117 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3b8627-c929-4ac9-8c39-eb10abca50e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:00.279136 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3b8627-c929-4ac9-8c39-eb10abca50e3 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:00.300312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1303/2598] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:56:00 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:56:00.304312 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f451a5e-1e11-46e6-a3f6-5fa2950c3039 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:00.315546 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f451a5e-1e11-46e6-a3f6-5fa2950c3039 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:00.320129 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dfa696-35a1-401c-9116-996295858eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:00.320787 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfa696-35a1-401c-9116-996295858eb7 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:00.321241 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfa696-35a1-401c-9116-996295858eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:00.321532 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfa696-35a1-401c-9116-996295858eb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:00.322265 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dfa696-35a1-401c-9116-996295858eb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:00.323864 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dfa696-35a1-401c-9116-996295858eb7 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:00.334311 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1296/2599] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:56:00.338399 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b8627-c929-4ac9-8c39-eb10abca50e3 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:00.348148 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b8627-c929-4ac9-8c39-eb10abca50e3 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:00.352662 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff75f9-b50d-4609-9a60-71f094d17ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:00.353585 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff75f9-b50d-4609-9a60-71f094d17ed7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:00.353867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff75f9-b50d-4609-9a60-71f094d17ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:00.354140 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff75f9-b50d-4609-9a60-71f094d17ed7 admin admin] PATH_INFO: `/v3/projects/e1dc9d1841c64313902db8c45251f05f/users/b926ec3ab1b7450dab5cff7dbc1ef455/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:00.369084 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ff75f9-b50d-4609-9a60-71f094d17ed7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1dc9d1841c64313902db8c45251f05f, user_id=b926ec3ab1b7450dab5cff7dbc1ef455, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:00.372357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ff75f9-b50d-4609-9a60-71f094d17ed7 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:00.385876 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1304/2600] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:56:00 2022] PUT /identity/v3/projects/e1dc9d1841c64313902db8c45251f05f/users/b926ec3ab1b7450dab5cff7dbc1ef455/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:56:00.389923 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dfa696-35a1-401c-9116-996295858eb7 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:00.399777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dfa696-35a1-401c-9116-996295858eb7 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:00.455050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf0b05a-d537-4ae8-8d6d-74670d7900f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:00.455688 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0b05a-d537-4ae8-8d6d-74670d7900f1 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:00.455913 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0b05a-d537-4ae8-8d6d-74670d7900f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:00.456193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0b05a-d537-4ae8-8d6d-74670d7900f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:00.456913 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf0b05a-d537-4ae8-8d6d-74670d7900f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:00.458533 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf0b05a-d537-4ae8-8d6d-74670d7900f1 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:00.468224 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1297/2601] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:56:00.472285 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff75f9-b50d-4609-9a60-71f094d17ed7 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:00.482992 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff75f9-b50d-4609-9a60-71f094d17ed7 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:00.487583 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c80871d-7a7c-4aa9-ba3a-fc6a00e4902d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:00.488452 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80871d-7a7c-4aa9-ba3a-fc6a00e4902d admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:00.488737 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80871d-7a7c-4aa9-ba3a-fc6a00e4902d admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:00.489039 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80871d-7a7c-4aa9-ba3a-fc6a00e4902d admin admin] PATH_INFO: `/v3/projects/e1dc9d1841c64313902db8c45251f05f/users/b926ec3ab1b7450dab5cff7dbc1ef455/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:00.490996 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c80871d-7a7c-4aa9-ba3a-fc6a00e4902d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1dc9d1841c64313902db8c45251f05f, user_id=b926ec3ab1b7450dab5cff7dbc1ef455, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:00.493639 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c80871d-7a7c-4aa9-ba3a-fc6a00e4902d admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:00.504244 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1305/2602] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:56:00 2022] PUT /identity/v3/projects/e1dc9d1841c64313902db8c45251f05f/users/b926ec3ab1b7450dab5cff7dbc1ef455/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:56:00.515291 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64553e8-e6f2-4b6f-8f71-f03f267fed35 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:00.515589 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64553e8-e6f2-4b6f-8f71-f03f267fed35 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:00.515873 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64553e8-e6f2-4b6f-8f71-f03f267fed35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:00.541406 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-a64553e8-e6f2-4b6f-8f71-f03f267fed35 None None] MFA Rules not processed for user `b926ec3ab1b7450dab5cff7dbc1ef455`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:56:00.542609 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a64553e8-e6f2-4b6f-8f71-f03f267fed35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:00.587157 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:56:00.614627 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1298/2603] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:56:00 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:56:00.626047 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c80871d-7a7c-4aa9-ba3a-fc6a00e4902d admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:00.627785 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-6c80871d-7a7c-4aa9-ba3a-fc6a00e4902d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:00.686322 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c80871d-7a7c-4aa9-ba3a-fc6a00e4902d admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:00.689981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d56353e-e112-4be5-8195-95fe5ea6dc26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:00.690626 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56353e-e112-4be5-8195-95fe5ea6dc26 service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:00.690906 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56353e-e112-4be5-8195-95fe5ea6dc26 service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:00.691183 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56353e-e112-4be5-8195-95fe5ea6dc26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:00.700326 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d56353e-e112-4be5-8195-95fe5ea6dc26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:00.701780 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d56353e-e112-4be5-8195-95fe5ea6dc26 service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:00.711919 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1306/2604] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:56:00 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:56:00.818232 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64553e8-e6f2-4b6f-8f71-f03f267fed35 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:00.827699 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64553e8-e6f2-4b6f-8f71-f03f267fed35 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:00.831147 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d2b6ac-039a-44ea-ba35-7a82d961076d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:00.831775 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2b6ac-039a-44ea-ba35-7a82d961076d service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:00.832055 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2b6ac-039a-44ea-ba35-7a82d961076d service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:00.832327 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2b6ac-039a-44ea-ba35-7a82d961076d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:00.886511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d2b6ac-039a-44ea-ba35-7a82d961076d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:00.888229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d2b6ac-039a-44ea-ba35-7a82d961076d service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:00.896844 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1299/2605] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:56:00 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:56:03.455949 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d56353e-e112-4be5-8195-95fe5ea6dc26 service cinder] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:03.489787 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:56:03.522708 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d56353e-e112-4be5-8195-95fe5ea6dc26 service cinder] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:03.527268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727d82c-c145-4256-a213-800df55c6265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:03.528943 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727d82c-c145-4256-a213-800df55c6265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:03.529259 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727d82c-c145-4256-a213-800df55c6265 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:03.529542 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727d82c-c145-4256-a213-800df55c6265 admin admin] PATH_INFO: `/v3/users/b889778476eb4b4eb316e16bf20415fe` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:03.530743 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8727d82c-c145-4256-a213-800df55c6265 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b889778476eb4b4eb316e16bf20415fe)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:03.532641 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8727d82c-c145-4256-a213-800df55c6265 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:03.578979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8727d82c-c145-4256-a213-800df55c6265 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b889778476eb4b4eb316e16bf20415fe'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:03.586309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8727d82c-c145-4256-a213-800df55c6265 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b889778476eb4b4eb316e16bf20415fe'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:03.594091 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8727d82c-c145-4256-a213-800df55c6265 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b889778476eb4b4eb316e16bf20415fe'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:03.594725 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8727d82c-c145-4256-a213-800df55c6265 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b889778476eb4b4eb316e16bf20415fe'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:03.606395 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8727d82c-c145-4256-a213-800df55c6265 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b889778476eb4b4eb316e16bf20415fe'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:03.612259 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1307/2606] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:56:03 2022] DELETE /identity/v3/users/b889778476eb4b4eb316e16bf20415fe => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:56:03.616742 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d086b-d943-4fd4-b84f-985bedfc1a99 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:03.617276 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d086b-d943-4fd4-b84f-985bedfc1a99 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:03.617398 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d086b-d943-4fd4-b84f-985bedfc1a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:03.634562 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-5a4d086b-d943-4fd4-b84f-985bedfc1a99 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:56:03.635740 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-5a4d086b-d943-4fd4-b84f-985bedfc1a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:03.693029 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1300/2607] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:56:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:56:03.703380 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727d82c-c145-4256-a213-800df55c6265 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:03.704787 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-8727d82c-c145-4256-a213-800df55c6265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:03.766897 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727d82c-c145-4256-a213-800df55c6265 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:03.770554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e458b-be93-4bbe-a572-ae37f8a6f423 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:03.771195 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e458b-be93-4bbe-a572-ae37f8a6f423 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:03.771481 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e458b-be93-4bbe-a572-ae37f8a6f423 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:03.771760 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e458b-be93-4bbe-a572-ae37f8a6f423 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:03.780685 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0e458b-be93-4bbe-a572-ae37f8a6f423 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:03.782136 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0e458b-be93-4bbe-a572-ae37f8a6f423 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:03.791519 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1308/2608] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:56:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:56:04.058317 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d086b-d943-4fd4-b84f-985bedfc1a99 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:04.060566 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-5a4d086b-d943-4fd4-b84f-985bedfc1a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:04.072825 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d086b-d943-4fd4-b84f-985bedfc1a99 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:04.077701 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88dd67-2fa1-4f45-9c62-3c267166f329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:04.078727 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88dd67-2fa1-4f45-9c62-3c267166f329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:04.079050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88dd67-2fa1-4f45-9c62-3c267166f329 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:04.079464 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88dd67-2fa1-4f45-9c62-3c267166f329 admin admin] PATH_INFO: `/v3/projects/ec0ddb31fecf45d29fad8dfd5efd6c23` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:04.080784 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88dd67-2fa1-4f45-9c62-3c267166f329 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec0ddb31fecf45d29fad8dfd5efd6c23)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:04.082792 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88dd67-2fa1-4f45-9c62-3c267166f329 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:04.138175 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-2e88dd67-2fa1-4f45-9c62-3c267166f329 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec0ddb31fecf45d29fad8dfd5efd6c23', 'request_id': 'req-2e88dd67-2fa1-4f45-9c62-3c267166f329'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:04.144800 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-2e88dd67-2fa1-4f45-9c62-3c267166f329 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec0ddb31fecf45d29fad8dfd5efd6c23', 'request_id': 'req-2e88dd67-2fa1-4f45-9c62-3c267166f329'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:04.152894 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-2e88dd67-2fa1-4f45-9c62-3c267166f329 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec0ddb31fecf45d29fad8dfd5efd6c23', 'request_id': 'req-2e88dd67-2fa1-4f45-9c62-3c267166f329'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:04.163749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-2e88dd67-2fa1-4f45-9c62-3c267166f329 admin admin] The token cache is being invalidate because project ec0ddb31fecf45d29fad8dfd5efd6c23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:56:04.164043 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-2e88dd67-2fa1-4f45-9c62-3c267166f329 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:04.164679 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-2e88dd67-2fa1-4f45-9c62-3c267166f329 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:04.166277 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1301/2609] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:56:04 2022] DELETE /identity/v3/projects/ec0ddb31fecf45d29fad8dfd5efd6c23 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:56:04.170937 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e458b-be93-4bbe-a572-ae37f8a6f423 service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:04.180723 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e458b-be93-4bbe-a572-ae37f8a6f423 service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:04.235551 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:56:04.245187 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4191713-3cb5-4167-9bce-bdaa3abf41cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:04.246069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4191713-3cb5-4167-9bce-bdaa3abf41cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:04.246351 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4191713-3cb5-4167-9bce-bdaa3abf41cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:04.246908 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4191713-3cb5-4167-9bce-bdaa3abf41cb admin admin] PATH_INFO: `/v3/users/b926ec3ab1b7450dab5cff7dbc1ef455` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:04.248271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4191713-3cb5-4167-9bce-bdaa3abf41cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b926ec3ab1b7450dab5cff7dbc1ef455)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:04.250197 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4191713-3cb5-4167-9bce-bdaa3abf41cb admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:04.301380 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b4191713-3cb5-4167-9bce-bdaa3abf41cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b926ec3ab1b7450dab5cff7dbc1ef455'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:04.308803 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b4191713-3cb5-4167-9bce-bdaa3abf41cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b926ec3ab1b7450dab5cff7dbc1ef455'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:04.316408 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b4191713-3cb5-4167-9bce-bdaa3abf41cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b926ec3ab1b7450dab5cff7dbc1ef455'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:04.317076 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b4191713-3cb5-4167-9bce-bdaa3abf41cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b926ec3ab1b7450dab5cff7dbc1ef455'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:04.329279 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b4191713-3cb5-4167-9bce-bdaa3abf41cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b926ec3ab1b7450dab5cff7dbc1ef455'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:04.335231 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1309/2610] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:56:04 2022] DELETE /identity/v3/users/b926ec3ab1b7450dab5cff7dbc1ef455 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:56:04.601221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88dd67-2fa1-4f45-9c62-3c267166f329 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:04.602671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-2e88dd67-2fa1-4f45-9c62-3c267166f329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:04.610911 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:56:04.619141 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88dd67-2fa1-4f45-9c62-3c267166f329 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:04.677812 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bdb38b-1b5c-44b1-8624-710f6256a893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:04.678673 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdb38b-1b5c-44b1-8624-710f6256a893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:04.678953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdb38b-1b5c-44b1-8624-710f6256a893 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:04.679226 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdb38b-1b5c-44b1-8624-710f6256a893 admin admin] PATH_INFO: `/v3/projects/e1dc9d1841c64313902db8c45251f05f` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:04.680381 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bdb38b-1b5c-44b1-8624-710f6256a893 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1dc9d1841c64313902db8c45251f05f)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:04.682301 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bdb38b-1b5c-44b1-8624-710f6256a893 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:04.733218 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-40bdb38b-1b5c-44b1-8624-710f6256a893 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1dc9d1841c64313902db8c45251f05f', 'request_id': 'req-40bdb38b-1b5c-44b1-8624-710f6256a893'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:04.739243 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-40bdb38b-1b5c-44b1-8624-710f6256a893 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1dc9d1841c64313902db8c45251f05f', 'request_id': 'req-40bdb38b-1b5c-44b1-8624-710f6256a893'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:04.747248 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-40bdb38b-1b5c-44b1-8624-710f6256a893 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1dc9d1841c64313902db8c45251f05f', 'request_id': 'req-40bdb38b-1b5c-44b1-8624-710f6256a893'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:04.758018 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-40bdb38b-1b5c-44b1-8624-710f6256a893 admin admin] The token cache is being invalidate because project e1dc9d1841c64313902db8c45251f05f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:56:04.758300 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-40bdb38b-1b5c-44b1-8624-710f6256a893 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:04.758912 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-40bdb38b-1b5c-44b1-8624-710f6256a893 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:04.760431 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1302/2611] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:56:04 2022] DELETE /identity/v3/projects/e1dc9d1841c64313902db8c45251f05f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:56:04.765879 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b346f-fd2b-4d61-9bac-e448530a003d None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:04.766378 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b346f-fd2b-4d61-9bac-e448530a003d None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:04.766488 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b346f-fd2b-4d61-9bac-e448530a003d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:04.783945 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-266b346f-fd2b-4d61-9bac-e448530a003d None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:56:04.785152 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-266b346f-fd2b-4d61-9bac-e448530a003d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:04.844795 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1310/2612] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:56:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:56:04.852679 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511c674-4a2c-4e56-ac38-cc9ca4e65485 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:04.852969 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511c674-4a2c-4e56-ac38-cc9ca4e65485 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:04.853272 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511c674-4a2c-4e56-ac38-cc9ca4e65485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:04.871365 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-5511c674-4a2c-4e56-ac38-cc9ca4e65485 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:56:04.873233 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-5511c674-4a2c-4e56-ac38-cc9ca4e65485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:04.880979 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1303/2613] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:56:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:56:04.885050 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b346f-fd2b-4d61-9bac-e448530a003d None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:04.894892 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b346f-fd2b-4d61-9bac-e448530a003d None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:04.899435 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76669186-8af4-4613-b86d-6254d3d05d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:04.900091 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76669186-8af4-4613-b86d-6254d3d05d58 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:04.900377 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76669186-8af4-4613-b86d-6254d3d05d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:04.900658 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76669186-8af4-4613-b86d-6254d3d05d58 admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:04.901550 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76669186-8af4-4613-b86d-6254d3d05d58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:56:04.901947 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76669186-8af4-4613-b86d-6254d3d05d58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:04.903820 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76669186-8af4-4613-b86d-6254d3d05d58 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:04.915339 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1311/2614] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:56:04.919772 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511c674-4a2c-4e56-ac38-cc9ca4e65485 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:04.929020 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511c674-4a2c-4e56-ac38-cc9ca4e65485 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:04.933860 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe3e3de-0717-448e-b1de-0408a3018337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:04.935148 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3e3de-0717-448e-b1de-0408a3018337 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:04.935433 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3e3de-0717-448e-b1de-0408a3018337 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:04.935712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3e3de-0717-448e-b1de-0408a3018337 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:04.936401 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe3e3de-0717-448e-b1de-0408a3018337 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:04.938321 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe3e3de-0717-448e-b1de-0408a3018337 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:04.964476 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1304/2615] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:56:04 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:56:04.969173 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76669186-8af4-4613-b86d-6254d3d05d58 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:04.978950 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76669186-8af4-4613-b86d-6254d3d05d58 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:05.036368 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666b711-21d1-4396-aeae-323501ad8d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:05.036954 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666b711-21d1-4396-aeae-323501ad8d41 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:05.037271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666b711-21d1-4396-aeae-323501ad8d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:05.037545 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666b711-21d1-4396-aeae-323501ad8d41 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:05.038233 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6666b711-21d1-4396-aeae-323501ad8d41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:05.040137 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6666b711-21d1-4396-aeae-323501ad8d41 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:05.047458 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:56:05.060508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1312/2616] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:56:04 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:56:05.064584 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe3e3de-0717-448e-b1de-0408a3018337 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:05.075377 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe3e3de-0717-448e-b1de-0408a3018337 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:05.080906 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5df61c-5f17-4779-821d-16ff2752284d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:05.081631 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5df61c-5f17-4779-821d-16ff2752284d admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:05.081948 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5df61c-5f17-4779-821d-16ff2752284d admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:05.082224 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5df61c-5f17-4779-821d-16ff2752284d admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:05.082946 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5df61c-5f17-4779-821d-16ff2752284d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:05.084563 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5df61c-5f17-4779-821d-16ff2752284d admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:05.095790 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1305/2617] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:56:05.099677 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666b711-21d1-4396-aeae-323501ad8d41 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:05.109486 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666b711-21d1-4396-aeae-323501ad8d41 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:05.114425 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc906a-f09b-4c5b-abf7-1178ec2da247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:05.115089 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc906a-f09b-4c5b-abf7-1178ec2da247 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:05.115374 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc906a-f09b-4c5b-abf7-1178ec2da247 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:05.115680 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc906a-f09b-4c5b-abf7-1178ec2da247 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:05.116408 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc906a-f09b-4c5b-abf7-1178ec2da247 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:05.118069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc906a-f09b-4c5b-abf7-1178ec2da247 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:05.128134 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1313/2618] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:56:05.132245 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5df61c-5f17-4779-821d-16ff2752284d admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:05.142695 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5df61c-5f17-4779-821d-16ff2752284d admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:05.147326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb53be-061d-4df6-b26a-c8debb2d6eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:05.148202 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb53be-061d-4df6-b26a-c8debb2d6eca admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:05.148486 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb53be-061d-4df6-b26a-c8debb2d6eca admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:05.148762 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb53be-061d-4df6-b26a-c8debb2d6eca admin admin] PATH_INFO: `/v3/projects/52dc411d6a7144889b45097422024c70/users/61dda783e9a74513819444fb64078cfb/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:05.163671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb53be-061d-4df6-b26a-c8debb2d6eca admin admin] RBAC: Authorizing `identity:create_grant(project_id=52dc411d6a7144889b45097422024c70, user_id=61dda783e9a74513819444fb64078cfb, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:05.166361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb53be-061d-4df6-b26a-c8debb2d6eca admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:05.179140 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1306/2619] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:56:05 2022] PUT /identity/v3/projects/52dc411d6a7144889b45097422024c70/users/61dda783e9a74513819444fb64078cfb/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:56:05.189859 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c5a89-2446-4791-8ad7-5905a91c1677 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:05.190148 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c5a89-2446-4791-8ad7-5905a91c1677 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:05.190421 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c5a89-2446-4791-8ad7-5905a91c1677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:05.217134 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-771c5a89-2446-4791-8ad7-5905a91c1677 None None] MFA Rules not processed for user `61dda783e9a74513819444fb64078cfb`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:56:05.218557 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-771c5a89-2446-4791-8ad7-5905a91c1677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:05.282413 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1314/2620] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:56:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:56:05.296003 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb53be-061d-4df6-b26a-c8debb2d6eca admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:05.297949 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-83bb53be-061d-4df6-b26a-c8debb2d6eca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:05.331874 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:56:05.358421 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb53be-061d-4df6-b26a-c8debb2d6eca admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:05.362207 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad95d5-bc3c-421a-9ea2-8b416cc1a455 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4015c5ca2b8a438288820cd003f9a308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:05.362852 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad95d5-bc3c-421a-9ea2-8b416cc1a455 service glance] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:05.363136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad95d5-bc3c-421a-9ea2-8b416cc1a455 service glance] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:05.363415 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad95d5-bc3c-421a-9ea2-8b416cc1a455 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:05.373549 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad95d5-bc3c-421a-9ea2-8b416cc1a455 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:05.374985 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad95d5-bc3c-421a-9ea2-8b416cc1a455 service glance] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:05.384278 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1307/2621] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:56:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:56:11.377482 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c5a89-2446-4791-8ad7-5905a91c1677 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:11.379430 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-771c5a89-2446-4791-8ad7-5905a91c1677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:11.460684 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c5a89-2446-4791-8ad7-5905a91c1677 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:11.472272 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72125e-1671-4ab7-88da-a16a45bb9b1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:11.473745 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72125e-1671-4ab7-88da-a16a45bb9b1e service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:11.475210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72125e-1671-4ab7-88da-a16a45bb9b1e service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:11.475480 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72125e-1671-4ab7-88da-a16a45bb9b1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:11.477934 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-ac72125e-1671-4ab7-88da-a16a45bb9b1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:11.553407 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac72125e-1671-4ab7-88da-a16a45bb9b1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:11.554948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac72125e-1671-4ab7-88da-a16a45bb9b1e service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:11.566158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1315/2622] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:56:11 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:56:15.380574 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=13 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:56:17.499350 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad95d5-bc3c-421a-9ea2-8b416cc1a455 service glance] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:17.500779 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-87ad95d5-bc3c-421a-9ea2-8b416cc1a455 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:17.561688 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad95d5-bc3c-421a-9ea2-8b416cc1a455 service glance] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:17.565256 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c358fe-b3c0-4837-8e7d-a9483b4acc5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:17.565898 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c358fe-b3c0-4837-8e7d-a9483b4acc5f service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:17.566177 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c358fe-b3c0-4837-8e7d-a9483b4acc5f service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:17.566453 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c358fe-b3c0-4837-8e7d-a9483b4acc5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:17.568205 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a9c358fe-b3c0-4837-8e7d-a9483b4acc5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:17.578259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c358fe-b3c0-4837-8e7d-a9483b4acc5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:17.579673 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c358fe-b3c0-4837-8e7d-a9483b4acc5f service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:17.588121 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1308/2623] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:56:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:56:21.561261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:INSERT=2,keystone:SELECT=87 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:56:23.868273 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f73e7-2854-4ba7-b661-6185266846b3 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:23.868842 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f73e7-2854-4ba7-b661-6185266846b3 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:23.868842 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f73e7-2854-4ba7-b661-6185266846b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:23.887735 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-888f73e7-2854-4ba7-b661-6185266846b3 None None] MFA Rules not processed for user `c4535134181e489fb532f617264d5855`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:56:23.889355 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-888f73e7-2854-4ba7-b661-6185266846b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:23.948773 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1316/2624] 10.222.0.20 () {58 vars in 967 bytes} [Wed Jul 6 13:56:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:56:23.972540 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c358fe-b3c0-4837-8e7d-a9483b4acc5f service neutron] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:23.973992 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a9c358fe-b3c0-4837-8e7d-a9483b4acc5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:24.047486 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c358fe-b3c0-4837-8e7d-a9483b4acc5f service neutron] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:24.053401 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846a8ea-ed79-4e42-85a8-9d7b980afc23 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c1bea05c37646b0a05fa8dda0367ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:24.054056 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846a8ea-ed79-4e42-85a8-9d7b980afc23 service swift] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:24.054344 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846a8ea-ed79-4e42-85a8-9d7b980afc23 service swift] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:24.054630 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846a8ea-ed79-4e42-85a8-9d7b980afc23 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:24.064105 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1846a8ea-ed79-4e42-85a8-9d7b980afc23 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:24.065734 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1846a8ea-ed79-4e42-85a8-9d7b980afc23 service swift] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:24.073975 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1309/2625] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jul 6 13:56:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 13:56:30.939595 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21a5d0-be34-4e0b-9b3c-0b60cdf4dcb9 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:30.940140 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21a5d0-be34-4e0b-9b3c-0b60cdf4dcb9 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:30.940140 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21a5d0-be34-4e0b-9b3c-0b60cdf4dcb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:30.958070 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-de21a5d0-be34-4e0b-9b3c-0b60cdf4dcb9 None None] MFA Rules not processed for user `32a8191b1c184c75b428579ace2da61c`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:56:30.959248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-de21a5d0-be34-4e0b-9b3c-0b60cdf4dcb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:30.966148 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1317/2626] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:56:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:56:31.093787 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846a8ea-ed79-4e42-85a8-9d7b980afc23 service swift] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:31.095297 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-1846a8ea-ed79-4e42-85a8-9d7b980afc23 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:31.160504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846a8ea-ed79-4e42-85a8-9d7b980afc23 service swift] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:31.165019 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5caca6-e4bb-4611-b9cc-4baf7142462a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:31.165890 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5caca6-e4bb-4611-b9cc-4baf7142462a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:31.166174 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5caca6-e4bb-4611-b9cc-4baf7142462a admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:31.166447 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5caca6-e4bb-4611-b9cc-4baf7142462a admin admin] PATH_INFO: `/v3/users/a75bdacc0fe5456cac89f8699bcf8f54` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:31.167622 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5caca6-e4bb-4611-b9cc-4baf7142462a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a75bdacc0fe5456cac89f8699bcf8f54)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:31.170097 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5caca6-e4bb-4611-b9cc-4baf7142462a admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:31.211200 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:56:31.221718 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-da5caca6-e4bb-4611-b9cc-4baf7142462a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a75bdacc0fe5456cac89f8699bcf8f54'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:31.229625 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-da5caca6-e4bb-4611-b9cc-4baf7142462a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a75bdacc0fe5456cac89f8699bcf8f54'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:31.230243 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-da5caca6-e4bb-4611-b9cc-4baf7142462a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a75bdacc0fe5456cac89f8699bcf8f54'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:31.238273 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-da5caca6-e4bb-4611-b9cc-4baf7142462a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a75bdacc0fe5456cac89f8699bcf8f54'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:31.251801 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-da5caca6-e4bb-4611-b9cc-4baf7142462a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a75bdacc0fe5456cac89f8699bcf8f54'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:31.257839 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1310/2627] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:56:31 2022] DELETE /identity/v3/users/a75bdacc0fe5456cac89f8699bcf8f54 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:56:31.262366 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37637d1f-7102-4fcb-b5f6-ca3c50625e30 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:31.262722 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37637d1f-7102-4fcb-b5f6-ca3c50625e30 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:31.263129 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37637d1f-7102-4fcb-b5f6-ca3c50625e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:31.280759 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-37637d1f-7102-4fcb-b5f6-ca3c50625e30 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:56:31.282108 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-37637d1f-7102-4fcb-b5f6-ca3c50625e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:31.349850 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1318/2628] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:56:31.356468 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5caca6-e4bb-4611-b9cc-4baf7142462a admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:31.358432 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-da5caca6-e4bb-4611-b9cc-4baf7142462a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:31.415830 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5caca6-e4bb-4611-b9cc-4baf7142462a admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:31.420263 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34679bf4-59fa-47bb-ae05-fa8c11989b1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:31.420920 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34679bf4-59fa-47bb-ae05-fa8c11989b1f service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:31.421243 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34679bf4-59fa-47bb-ae05-fa8c11989b1f service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:31.421518 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34679bf4-59fa-47bb-ae05-fa8c11989b1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:31.431118 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34679bf4-59fa-47bb-ae05-fa8c11989b1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:31.432536 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34679bf4-59fa-47bb-ae05-fa8c11989b1f service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:31.442004 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1311/2629] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:56:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:56:31.721822 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37637d1f-7102-4fcb-b5f6-ca3c50625e30 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:31.723676 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-37637d1f-7102-4fcb-b5f6-ca3c50625e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:31.755334 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37637d1f-7102-4fcb-b5f6-ca3c50625e30 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:31.765803 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e6c0c-a668-4663-912f-7728fcfe577c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:31.766659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e6c0c-a668-4663-912f-7728fcfe577c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:31.766940 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e6c0c-a668-4663-912f-7728fcfe577c admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:31.767215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e6c0c-a668-4663-912f-7728fcfe577c admin admin] PATH_INFO: `/v3/projects/093c882a86024b269c1f90973f9ddcf1` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:31.769151 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535e6c0c-a668-4663-912f-7728fcfe577c admin admin] RBAC: Authorizing `identity:delete_project(project_id=093c882a86024b269c1f90973f9ddcf1)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:31.771197 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535e6c0c-a668-4663-912f-7728fcfe577c admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:31.792339 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34679bf4-59fa-47bb-ae05-fa8c11989b1f service neutron] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:31.793919 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-34679bf4-59fa-47bb-ae05-fa8c11989b1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:31.828791 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-535e6c0c-a668-4663-912f-7728fcfe577c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '093c882a86024b269c1f90973f9ddcf1', 'request_id': 'req-535e6c0c-a668-4663-912f-7728fcfe577c'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:31.835217 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-535e6c0c-a668-4663-912f-7728fcfe577c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '093c882a86024b269c1f90973f9ddcf1', 'request_id': 'req-535e6c0c-a668-4663-912f-7728fcfe577c'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:31.845049 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-535e6c0c-a668-4663-912f-7728fcfe577c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '093c882a86024b269c1f90973f9ddcf1', 'request_id': 'req-535e6c0c-a668-4663-912f-7728fcfe577c'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:31.860034 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-535e6c0c-a668-4663-912f-7728fcfe577c admin admin] The token cache is being invalidate because project 093c882a86024b269c1f90973f9ddcf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:56:31.860292 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-535e6c0c-a668-4663-912f-7728fcfe577c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:31.860896 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-535e6c0c-a668-4663-912f-7728fcfe577c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:31.862762 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1319/2630] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:56:31 2022] DELETE /identity/v3/projects/093c882a86024b269c1f90973f9ddcf1 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:56:31.867525 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f7ca7-223e-451c-9256-4d33e9b117c4 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:31.867818 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f7ca7-223e-451c-9256-4d33e9b117c4 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:31.868105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f7ca7-223e-451c-9256-4d33e9b117c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:31.884109 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34679bf4-59fa-47bb-ae05-fa8c11989b1f service neutron] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:31.889700 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-a31f7ca7-223e-451c-9256-4d33e9b117c4 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:56:31.890584 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-a31f7ca7-223e-451c-9256-4d33e9b117c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:31.914059 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:56:31.955679 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:56:31.969046 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1320/2631] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:56:31.975464 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42cb99b-b265-4410-b897-e1759e7a6567 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '890d2ae150d647a18dae25159cf97484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:31.976599 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cb99b-b265-4410-b897-e1759e7a6567 service placement] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:31.976916 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cb99b-b265-4410-b897-e1759e7a6567 service placement] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:31.977227 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cb99b-b265-4410-b897-e1759e7a6567 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:31.977471 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5fcfd9-9024-4db7-b5da-8fe602e57106 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:31.977919 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5fcfd9-9024-4db7-b5da-8fe602e57106 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:31.978485 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5fcfd9-9024-4db7-b5da-8fe602e57106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:31.979253 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-b42cb99b-b265-4410-b897-e1759e7a6567 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:31.992044 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42cb99b-b265-4410-b897-e1759e7a6567 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:31.994001 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42cb99b-b265-4410-b897-e1759e7a6567 service placement] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:32.002181 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-8d5fcfd9-9024-4db7-b5da-8fe602e57106 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:56:32.003730 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-8d5fcfd9-9024-4db7-b5da-8fe602e57106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:32.006194 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1312/2632] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:56:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 13:56:32.013039 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1321/2633] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:56:32.016400 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42cb99b-b265-4410-b897-e1759e7a6567 service placement] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:32.027182 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42cb99b-b265-4410-b897-e1759e7a6567 service placement] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:32.033707 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e1ef31-b450-4a36-bf3d-9b4bca3b9cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:32.034633 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e1ef31-b450-4a36-bf3d-9b4bca3b9cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:32.035040 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e1ef31-b450-4a36-bf3d-9b4bca3b9cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:32.035418 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e1ef31-b450-4a36-bf3d-9b4bca3b9cc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:32.036251 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e1ef31-b450-4a36-bf3d-9b4bca3b9cc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:56:32.036660 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e1ef31-b450-4a36-bf3d-9b4bca3b9cc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:32.038429 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e1ef31-b450-4a36-bf3d-9b4bca3b9cc0 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:32.057658 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1313/2634] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:56:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:56:32.058775 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5fcfd9-9024-4db7-b5da-8fe602e57106 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:32.077596 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5fcfd9-9024-4db7-b5da-8fe602e57106 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:32.086792 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ad93b-8d13-409e-a458-26ac5ad5a884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:32.087603 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ad93b-8d13-409e-a458-26ac5ad5a884 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:32.088136 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ad93b-8d13-409e-a458-26ac5ad5a884 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:32.088486 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ad93b-8d13-409e-a458-26ac5ad5a884 admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:32.089351 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7ad93b-8d13-409e-a458-26ac5ad5a884 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:32.092409 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7ad93b-8d13-409e-a458-26ac5ad5a884 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:32.131033 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1322/2635] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:56:32 2022] POST /identity/v3/projects => generated 373 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:56:32.131999 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e1ef31-b450-4a36-bf3d-9b4bca3b9cc0 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:32.145664 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e1ef31-b450-4a36-bf3d-9b4bca3b9cc0 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:32.206483 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994bcd2e-02fc-4da5-ac7f-6fda5e3d4f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:32.207161 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994bcd2e-02fc-4da5-ac7f-6fda5e3d4f74 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:32.207484 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994bcd2e-02fc-4da5-ac7f-6fda5e3d4f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:32.207782 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994bcd2e-02fc-4da5-ac7f-6fda5e3d4f74 admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:32.208428 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994bcd2e-02fc-4da5-ac7f-6fda5e3d4f74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:32.210434 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994bcd2e-02fc-4da5-ac7f-6fda5e3d4f74 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:32.238407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1314/2636] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:56:32 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:56:32.240465 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ad93b-8d13-409e-a458-26ac5ad5a884 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:32.251116 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ad93b-8d13-409e-a458-26ac5ad5a884 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:32.256425 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379895e-8931-417b-9efc-39dc0e7497bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:32.257346 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379895e-8931-417b-9efc-39dc0e7497bf admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:32.257723 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379895e-8931-417b-9efc-39dc0e7497bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:32.258072 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379895e-8931-417b-9efc-39dc0e7497bf admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:32.259199 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e379895e-8931-417b-9efc-39dc0e7497bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:32.260570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e379895e-8931-417b-9efc-39dc0e7497bf admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:32.275674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1323/2637] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:56:32.275971 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994bcd2e-02fc-4da5-ac7f-6fda5e3d4f74 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:32.286943 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994bcd2e-02fc-4da5-ac7f-6fda5e3d4f74 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:32.297623 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab109aac-0758-4066-a83e-8f28a79375f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:32.298183 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab109aac-0758-4066-a83e-8f28a79375f9 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:32.298856 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab109aac-0758-4066-a83e-8f28a79375f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:32.299293 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab109aac-0758-4066-a83e-8f28a79375f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:32.300077 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab109aac-0758-4066-a83e-8f28a79375f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:32.302392 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab109aac-0758-4066-a83e-8f28a79375f9 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:32.314034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1315/2638] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:56:32.318077 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379895e-8931-417b-9efc-39dc0e7497bf admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:32.329163 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379895e-8931-417b-9efc-39dc0e7497bf admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:32.334471 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5c111-166e-46e0-9070-d3a00acb3b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:32.335756 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5c111-166e-46e0-9070-d3a00acb3b7a admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:32.336046 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5c111-166e-46e0-9070-d3a00acb3b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:32.336323 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5c111-166e-46e0-9070-d3a00acb3b7a admin admin] PATH_INFO: `/v3/projects/4ee9bd87050f47009e091f72fec1688e/users/2fa3464fa21e456ebf42fde96ef1b9a6/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:32.352707 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e5c111-166e-46e0-9070-d3a00acb3b7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ee9bd87050f47009e091f72fec1688e, user_id=2fa3464fa21e456ebf42fde96ef1b9a6, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:32.356042 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e5c111-166e-46e0-9070-d3a00acb3b7a admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:32.372146 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1324/2639] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:56:32 2022] PUT /identity/v3/projects/4ee9bd87050f47009e091f72fec1688e/users/2fa3464fa21e456ebf42fde96ef1b9a6/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:56:32.382888 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc0767-9afe-4c1f-931b-da8142e18845 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:32.383186 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc0767-9afe-4c1f-931b-da8142e18845 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:32.383472 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc0767-9afe-4c1f-931b-da8142e18845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:32.411177 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-9edc0767-9afe-4c1f-931b-da8142e18845 None None] MFA Rules not processed for user `2fa3464fa21e456ebf42fde96ef1b9a6`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:56:32.412402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-9edc0767-9afe-4c1f-931b-da8142e18845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:32.478111 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1316/2640] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:56:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:56:32.489325 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5c111-166e-46e0-9070-d3a00acb3b7a admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:32.491162 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-c4e5c111-166e-46e0-9070-d3a00acb3b7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:32.551785 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5c111-166e-46e0-9070-d3a00acb3b7a admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:32.556026 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b8780-3fa8-4c13-a0f0-b5ff17da1333 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:32.556805 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b8780-3fa8-4c13-a0f0-b5ff17da1333 service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:32.557048 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b8780-3fa8-4c13-a0f0-b5ff17da1333 service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:32.557394 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b8780-3fa8-4c13-a0f0-b5ff17da1333 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:32.567308 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b8780-3fa8-4c13-a0f0-b5ff17da1333 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:32.568742 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b8780-3fa8-4c13-a0f0-b5ff17da1333 service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:32.578756 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1325/2641] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:56:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:56:34.504610 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc0767-9afe-4c1f-931b-da8142e18845 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:34.506168 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-9edc0767-9afe-4c1f-931b-da8142e18845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:34.575169 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:56:34.615969 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc0767-9afe-4c1f-931b-da8142e18845 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:34.622019 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5273e2-8d89-4f99-ab26-f4b8c04a3b35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:34.623732 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5273e2-8d89-4f99-ab26-f4b8c04a3b35 service nova] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:34.624132 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5273e2-8d89-4f99-ab26-f4b8c04a3b35 service nova] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:34.624414 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5273e2-8d89-4f99-ab26-f4b8c04a3b35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:34.626305 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-cb5273e2-8d89-4f99-ab26-f4b8c04a3b35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:34.640600 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5273e2-8d89-4f99-ab26-f4b8c04a3b35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:34.642243 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5273e2-8d89-4f99-ab26-f4b8c04a3b35 service nova] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:34.654859 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1317/2642] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:56:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:56:36.637431 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b8780-3fa8-4c13-a0f0-b5ff17da1333 service cinder] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:36.701712 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b8780-3fa8-4c13-a0f0-b5ff17da1333 service cinder] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:36.706277 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6523f239-0b91-4a10-bd5a-284187c2383e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:36.707178 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523f239-0b91-4a10-bd5a-284187c2383e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:36.707423 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523f239-0b91-4a10-bd5a-284187c2383e admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:36.707712 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523f239-0b91-4a10-bd5a-284187c2383e admin admin] PATH_INFO: `/v3/users/2fa3464fa21e456ebf42fde96ef1b9a6` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:36.708906 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6523f239-0b91-4a10-bd5a-284187c2383e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fa3464fa21e456ebf42fde96ef1b9a6)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:36.710863 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6523f239-0b91-4a10-bd5a-284187c2383e admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:36.713326 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:56:36.764040 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-6523f239-0b91-4a10-bd5a-284187c2383e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fa3464fa21e456ebf42fde96ef1b9a6'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:36.772615 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-6523f239-0b91-4a10-bd5a-284187c2383e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fa3464fa21e456ebf42fde96ef1b9a6'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:36.779957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-6523f239-0b91-4a10-bd5a-284187c2383e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fa3464fa21e456ebf42fde96ef1b9a6'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:36.780589 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-6523f239-0b91-4a10-bd5a-284187c2383e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fa3464fa21e456ebf42fde96ef1b9a6'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:36.792108 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-6523f239-0b91-4a10-bd5a-284187c2383e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fa3464fa21e456ebf42fde96ef1b9a6'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:36.797927 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1326/2643] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:56:36 2022] DELETE /identity/v3/users/2fa3464fa21e456ebf42fde96ef1b9a6 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:56:36.802514 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2fc13-b9d8-42fc-9eed-d5751d0e38fd None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:36.802973 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2fc13-b9d8-42fc-9eed-d5751d0e38fd None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:36.803098 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2fc13-b9d8-42fc-9eed-d5751d0e38fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:36.820869 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-2ab2fc13-b9d8-42fc-9eed-d5751d0e38fd None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:56:36.822073 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-2ab2fc13-b9d8-42fc-9eed-d5751d0e38fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:36.880832 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1318/2644] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:56:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:56:36.890244 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6523f239-0b91-4a10-bd5a-284187c2383e admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:36.891644 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-6523f239-0b91-4a10-bd5a-284187c2383e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:36.949072 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6523f239-0b91-4a10-bd5a-284187c2383e admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:36.952749 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc190cec-ca0c-4829-b770-022bd6429bfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:36.953429 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc190cec-ca0c-4829-b770-022bd6429bfe service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:36.953717 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc190cec-ca0c-4829-b770-022bd6429bfe service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:36.953998 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc190cec-ca0c-4829-b770-022bd6429bfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:36.962845 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc190cec-ca0c-4829-b770-022bd6429bfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:36.964266 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc190cec-ca0c-4829-b770-022bd6429bfe service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:36.974293 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1327/2645] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:56:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:56:37.245204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2fc13-b9d8-42fc-9eed-d5751d0e38fd None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:37.246667 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-2ab2fc13-b9d8-42fc-9eed-d5751d0e38fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:37.265614 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2fc13-b9d8-42fc-9eed-d5751d0e38fd None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:37.271201 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938edfe-cfb3-42ca-9779-d1e7975c456e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:37.272276 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938edfe-cfb3-42ca-9779-d1e7975c456e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:37.272704 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938edfe-cfb3-42ca-9779-d1e7975c456e admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:37.273062 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938edfe-cfb3-42ca-9779-d1e7975c456e admin admin] PATH_INFO: `/v3/projects/4ee9bd87050f47009e091f72fec1688e` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:37.274500 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b938edfe-cfb3-42ca-9779-d1e7975c456e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ee9bd87050f47009e091f72fec1688e)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:37.276489 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b938edfe-cfb3-42ca-9779-d1e7975c456e admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:37.341152 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-b938edfe-cfb3-42ca-9779-d1e7975c456e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ee9bd87050f47009e091f72fec1688e', 'request_id': 'req-b938edfe-cfb3-42ca-9779-d1e7975c456e'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:37.348755 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-b938edfe-cfb3-42ca-9779-d1e7975c456e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ee9bd87050f47009e091f72fec1688e', 'request_id': 'req-b938edfe-cfb3-42ca-9779-d1e7975c456e'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:37.356625 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-b938edfe-cfb3-42ca-9779-d1e7975c456e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ee9bd87050f47009e091f72fec1688e', 'request_id': 'req-b938edfe-cfb3-42ca-9779-d1e7975c456e'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:37.367080 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-b938edfe-cfb3-42ca-9779-d1e7975c456e admin admin] The token cache is being invalidate because project 4ee9bd87050f47009e091f72fec1688e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:56:37.367368 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-b938edfe-cfb3-42ca-9779-d1e7975c456e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:37.367992 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-b938edfe-cfb3-42ca-9779-d1e7975c456e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:37.369574 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1319/2646] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:56:37 2022] DELETE /identity/v3/projects/4ee9bd87050f47009e091f72fec1688e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:56:37.374978 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849ef0c-33ab-4389-8cdd-efb9590d9bba None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:37.374978 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849ef0c-33ab-4389-8cdd-efb9590d9bba None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:37.375359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849ef0c-33ab-4389-8cdd-efb9590d9bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:37.395833 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-9849ef0c-33ab-4389-8cdd-efb9590d9bba None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:56:37.396870 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-9849ef0c-33ab-4389-8cdd-efb9590d9bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:37.465546 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1328/2647] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:56:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:56:37.474436 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef61ca-bdef-483e-bac3-6867e15a1633 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:37.474726 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef61ca-bdef-483e-bac3-6867e15a1633 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:37.475001 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef61ca-bdef-483e-bac3-6867e15a1633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:37.499624 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-94ef61ca-bdef-483e-bac3-6867e15a1633 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:56:37.501732 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-94ef61ca-bdef-483e-bac3-6867e15a1633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:37.510277 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1320/2648] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:56:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:56:37.515433 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9849ef0c-33ab-4389-8cdd-efb9590d9bba None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:37.525502 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9849ef0c-33ab-4389-8cdd-efb9590d9bba None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:37.531145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eaf905-2d40-4f5f-8cfd-b0ad51158108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:37.531809 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eaf905-2d40-4f5f-8cfd-b0ad51158108 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:37.532093 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eaf905-2d40-4f5f-8cfd-b0ad51158108 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:37.532369 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eaf905-2d40-4f5f-8cfd-b0ad51158108 admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:37.533133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eaf905-2d40-4f5f-8cfd-b0ad51158108 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:56:37.533442 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eaf905-2d40-4f5f-8cfd-b0ad51158108 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:37.535090 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eaf905-2d40-4f5f-8cfd-b0ad51158108 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:37.547492 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1329/2649] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:56:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:56:37.552174 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef61ca-bdef-483e-bac3-6867e15a1633 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:37.562198 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef61ca-bdef-483e-bac3-6867e15a1633 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:37.566820 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a8597-af3f-4862-9ae3-9146ecc0512f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:37.567480 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a8597-af3f-4862-9ae3-9146ecc0512f admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:37.567763 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a8597-af3f-4862-9ae3-9146ecc0512f admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:37.568041 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a8597-af3f-4862-9ae3-9146ecc0512f admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:37.568732 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a8597-af3f-4862-9ae3-9146ecc0512f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:37.570674 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a8597-af3f-4862-9ae3-9146ecc0512f admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:37.600267 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1321/2650] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:56:37 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:56:37.601333 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eaf905-2d40-4f5f-8cfd-b0ad51158108 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:37.613188 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eaf905-2d40-4f5f-8cfd-b0ad51158108 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:37.621160 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:56:37.677057 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5667abf3-e43e-4092-8696-e1c7a3cda9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:37.677716 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667abf3-e43e-4092-8696-e1c7a3cda9d2 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:37.677958 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667abf3-e43e-4092-8696-e1c7a3cda9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:37.678233 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667abf3-e43e-4092-8696-e1c7a3cda9d2 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:37.678935 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5667abf3-e43e-4092-8696-e1c7a3cda9d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:37.680833 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5667abf3-e43e-4092-8696-e1c7a3cda9d2 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:37.702157 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1330/2651] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:56:37 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:56:37.706270 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a8597-af3f-4862-9ae3-9146ecc0512f admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:37.717338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a8597-af3f-4862-9ae3-9146ecc0512f admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:37.721958 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d394f152-3e5e-4bcb-b16d-a33d63177b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:37.722737 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394f152-3e5e-4bcb-b16d-a33d63177b63 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:37.723032 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394f152-3e5e-4bcb-b16d-a33d63177b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:37.723312 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394f152-3e5e-4bcb-b16d-a33d63177b63 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:37.724038 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d394f152-3e5e-4bcb-b16d-a33d63177b63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:37.725678 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d394f152-3e5e-4bcb-b16d-a33d63177b63 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:37.736814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1322/2652] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:56:37.741280 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5667abf3-e43e-4092-8696-e1c7a3cda9d2 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:37.751881 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5667abf3-e43e-4092-8696-e1c7a3cda9d2 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:37.756772 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada9ba7-76a0-4506-8b76-e37f3a06f373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:37.757541 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada9ba7-76a0-4506-8b76-e37f3a06f373 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:37.757827 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada9ba7-76a0-4506-8b76-e37f3a06f373 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:37.758104 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada9ba7-76a0-4506-8b76-e37f3a06f373 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:37.758829 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ada9ba7-76a0-4506-8b76-e37f3a06f373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:37.760408 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ada9ba7-76a0-4506-8b76-e37f3a06f373 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:37.771311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1331/2653] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:56:37.775466 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d394f152-3e5e-4bcb-b16d-a33d63177b63 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:37.786218 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d394f152-3e5e-4bcb-b16d-a33d63177b63 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:37.790855 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabfe006-bc0d-46fe-b574-34f9d26d30ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:37.791736 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfe006-bc0d-46fe-b574-34f9d26d30ca admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:37.792047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfe006-bc0d-46fe-b574-34f9d26d30ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:37.792327 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfe006-bc0d-46fe-b574-34f9d26d30ca admin admin] PATH_INFO: `/v3/projects/5486bfe046db4a7e883a6307e0c0f61c/users/a5bf18812aaf4f4083d8f3aa32a1f0a6/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:37.802028 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:56:37.811550 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabfe006-bc0d-46fe-b574-34f9d26d30ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=5486bfe046db4a7e883a6307e0c0f61c, user_id=a5bf18812aaf4f4083d8f3aa32a1f0a6, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:37.814900 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabfe006-bc0d-46fe-b574-34f9d26d30ca admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:37.830244 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1323/2654] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:56:37 2022] PUT /identity/v3/projects/5486bfe046db4a7e883a6307e0c0f61c/users/a5bf18812aaf4f4083d8f3aa32a1f0a6/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:56:37.846080 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded8c7d-5900-4057-ac3b-62467b8583d0 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:37.846391 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded8c7d-5900-4057-ac3b-62467b8583d0 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:37.846677 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded8c7d-5900-4057-ac3b-62467b8583d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:37.878145 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-fded8c7d-5900-4057-ac3b-62467b8583d0 None None] MFA Rules not processed for user `a5bf18812aaf4f4083d8f3aa32a1f0a6`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:56:37.879341 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-fded8c7d-5900-4057-ac3b-62467b8583d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:37.955075 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1332/2655] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:56:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:56:37.959532 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabfe006-bc0d-46fe-b574-34f9d26d30ca admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:38.021923 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabfe006-bc0d-46fe-b574-34f9d26d30ca admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:38.026742 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6317fe-442f-4f5c-b4ac-5501605156a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:38.027393 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6317fe-442f-4f5c-b4ac-5501605156a5 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:38.027676 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6317fe-442f-4f5c-b4ac-5501605156a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:38.027951 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6317fe-442f-4f5c-b4ac-5501605156a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:38.029628 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6317fe-442f-4f5c-b4ac-5501605156a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:38.031502 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6317fe-442f-4f5c-b4ac-5501605156a5 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:38.056537 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1324/2656] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:56:37 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:56:38.061123 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded8c7d-5900-4057-ac3b-62467b8583d0 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:38.071636 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded8c7d-5900-4057-ac3b-62467b8583d0 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:38.130133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf14e9-a8e1-4e8c-af90-2de9a72bca81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:38.130800 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf14e9-a8e1-4e8c-af90-2de9a72bca81 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:38.131082 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf14e9-a8e1-4e8c-af90-2de9a72bca81 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:38.131359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf14e9-a8e1-4e8c-af90-2de9a72bca81 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:38.132037 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcf14e9-a8e1-4e8c-af90-2de9a72bca81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:38.133980 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcf14e9-a8e1-4e8c-af90-2de9a72bca81 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:38.157106 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1333/2657] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:56:38 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:56:38.161136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6317fe-442f-4f5c-b4ac-5501605156a5 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:38.171455 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6317fe-442f-4f5c-b4ac-5501605156a5 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:38.176273 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de50bc1-557b-4630-836b-d07347b18bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:38.176926 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de50bc1-557b-4630-836b-d07347b18bcc admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:38.177245 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de50bc1-557b-4630-836b-d07347b18bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:38.177523 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de50bc1-557b-4630-836b-d07347b18bcc admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:38.178268 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de50bc1-557b-4630-836b-d07347b18bcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:38.179857 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de50bc1-557b-4630-836b-d07347b18bcc admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:38.190863 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1325/2658] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:56:38.194650 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf14e9-a8e1-4e8c-af90-2de9a72bca81 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:38.205788 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf14e9-a8e1-4e8c-af90-2de9a72bca81 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:38.210311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed608aa-b193-4da3-9e28-81a4f9a1078a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:38.210959 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed608aa-b193-4da3-9e28-81a4f9a1078a admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:38.211239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed608aa-b193-4da3-9e28-81a4f9a1078a admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:38.211510 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed608aa-b193-4da3-9e28-81a4f9a1078a admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:38.212226 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed608aa-b193-4da3-9e28-81a4f9a1078a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:38.213839 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed608aa-b193-4da3-9e28-81a4f9a1078a admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:38.215776 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:56:38.225781 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1334/2659] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:56:38.229574 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de50bc1-557b-4630-836b-d07347b18bcc admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:38.239745 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de50bc1-557b-4630-836b-d07347b18bcc admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:38.244354 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62326cf8-6eac-48a1-b690-6685d8b53958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:38.245266 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62326cf8-6eac-48a1-b690-6685d8b53958 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:38.245546 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62326cf8-6eac-48a1-b690-6685d8b53958 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:38.245821 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62326cf8-6eac-48a1-b690-6685d8b53958 admin admin] PATH_INFO: `/v3/projects/ffae6d693de743fe96d94283d2e22ba6/users/c87d1d2c23be4a38a410ddc72d27ebcb/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:38.261522 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62326cf8-6eac-48a1-b690-6685d8b53958 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffae6d693de743fe96d94283d2e22ba6, user_id=c87d1d2c23be4a38a410ddc72d27ebcb, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:38.264889 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62326cf8-6eac-48a1-b690-6685d8b53958 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:38.278271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1326/2660] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:56:38 2022] PUT /identity/v3/projects/ffae6d693de743fe96d94283d2e22ba6/users/c87d1d2c23be4a38a410ddc72d27ebcb/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 13:56:38.288212 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e6b43-7240-4e27-b8ca-4f957537e6a4 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:38.288505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e6b43-7240-4e27-b8ca-4f957537e6a4 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:38.288788 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e6b43-7240-4e27-b8ca-4f957537e6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:38.317866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-9a7e6b43-7240-4e27-b8ca-4f957537e6a4 None None] MFA Rules not processed for user `c87d1d2c23be4a38a410ddc72d27ebcb`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:56:38.319046 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-9a7e6b43-7240-4e27-b8ca-4f957537e6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:38.384334 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1335/2661] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:56:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:56:38.389640 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62326cf8-6eac-48a1-b690-6685d8b53958 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:38.458930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62326cf8-6eac-48a1-b690-6685d8b53958 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:38.463582 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf522b2-aafe-4784-b9cb-75b541abe6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:38.464245 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf522b2-aafe-4784-b9cb-75b541abe6ca admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:38.464532 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf522b2-aafe-4784-b9cb-75b541abe6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:38.464814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf522b2-aafe-4784-b9cb-75b541abe6ca admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:38.465541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf522b2-aafe-4784-b9cb-75b541abe6ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:38.467679 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf522b2-aafe-4784-b9cb-75b541abe6ca admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:38.492850 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1327/2662] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:56:38 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:56:38.497109 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7e6b43-7240-4e27-b8ca-4f957537e6a4 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:38.508359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7e6b43-7240-4e27-b8ca-4f957537e6a4 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:38.564632 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013fc2a-b360-48d1-938d-de6944f5aabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:38.565399 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013fc2a-b360-48d1-938d-de6944f5aabf admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:38.565682 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013fc2a-b360-48d1-938d-de6944f5aabf admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:38.565956 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013fc2a-b360-48d1-938d-de6944f5aabf admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:38.566649 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9013fc2a-b360-48d1-938d-de6944f5aabf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:38.568541 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9013fc2a-b360-48d1-938d-de6944f5aabf admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:38.587070 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1336/2663] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:56:38 2022] POST /identity/v3/users => generated 295 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:56:38.591141 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf522b2-aafe-4784-b9cb-75b541abe6ca admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:38.601216 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf522b2-aafe-4784-b9cb-75b541abe6ca admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:38.605752 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4188e-cb3b-4a95-886b-d7ac3174f7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:38.606402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4188e-cb3b-4a95-886b-d7ac3174f7f7 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:38.607202 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4188e-cb3b-4a95-886b-d7ac3174f7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:38.607653 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4188e-cb3b-4a95-886b-d7ac3174f7f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:38.608423 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a4188e-cb3b-4a95-886b-d7ac3174f7f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:38.610044 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a4188e-cb3b-4a95-886b-d7ac3174f7f7 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:38.620263 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1328/2664] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:56:38.626582 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013fc2a-b360-48d1-938d-de6944f5aabf admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:38.636578 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013fc2a-b360-48d1-938d-de6944f5aabf admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:38.641414 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f9b4a-aade-443b-8781-9632137d89a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:38.642287 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f9b4a-aade-443b-8781-9632137d89a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:38.642568 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f9b4a-aade-443b-8781-9632137d89a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:38.642842 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f9b4a-aade-443b-8781-9632137d89a5 admin admin] PATH_INFO: `/v3/projects/9a0c8ea7168f40c7bb988fb895704c14/users/d73eca723c7547b48cf1ed7d5bc0cae5/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:38.657442 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4f9b4a-aade-443b-8781-9632137d89a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a0c8ea7168f40c7bb988fb895704c14, user_id=d73eca723c7547b48cf1ed7d5bc0cae5, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:38.660079 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4f9b4a-aade-443b-8781-9632137d89a5 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:38.672742 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1337/2665] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:56:38 2022] PUT /identity/v3/projects/9a0c8ea7168f40c7bb988fb895704c14/users/d73eca723c7547b48cf1ed7d5bc0cae5/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:56:38.676892 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4188e-cb3b-4a95-886b-d7ac3174f7f7 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:38.687984 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4188e-cb3b-4a95-886b-d7ac3174f7f7 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:38.695717 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:56:38.749633 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d495099-85c7-4edb-bb56-a1aa704bc5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:38.750364 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d495099-85c7-4edb-bb56-a1aa704bc5d5 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:38.750664 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d495099-85c7-4edb-bb56-a1aa704bc5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:38.750937 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d495099-85c7-4edb-bb56-a1aa704bc5d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:38.751665 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d495099-85c7-4edb-bb56-a1aa704bc5d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:38.753274 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d495099-85c7-4edb-bb56-a1aa704bc5d5 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:38.763646 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1329/2666] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:56:38.767751 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f9b4a-aade-443b-8781-9632137d89a5 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:38.778618 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f9b4a-aade-443b-8781-9632137d89a5 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:38.783704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3947ad-53a5-4b7d-83b2-833a9ad52d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:38.784815 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3947ad-53a5-4b7d-83b2-833a9ad52d26 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:38.785313 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3947ad-53a5-4b7d-83b2-833a9ad52d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:38.785696 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3947ad-53a5-4b7d-83b2-833a9ad52d26 admin admin] PATH_INFO: `/v3/projects/9a0c8ea7168f40c7bb988fb895704c14/users/d73eca723c7547b48cf1ed7d5bc0cae5/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:38.788135 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3947ad-53a5-4b7d-83b2-833a9ad52d26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a0c8ea7168f40c7bb988fb895704c14, user_id=d73eca723c7547b48cf1ed7d5bc0cae5, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:38.790891 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3947ad-53a5-4b7d-83b2-833a9ad52d26 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:38.803703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1338/2667] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:56:38 2022] PUT /identity/v3/projects/9a0c8ea7168f40c7bb988fb895704c14/users/d73eca723c7547b48cf1ed7d5bc0cae5/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:56:38.870867 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06feaf37-ae52-47f0-b63c-904478de6770 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:38.871406 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06feaf37-ae52-47f0-b63c-904478de6770 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:38.871406 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06feaf37-ae52-47f0-b63c-904478de6770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:38.899848 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-06feaf37-ae52-47f0-b63c-904478de6770 None None] MFA Rules not processed for user `d73eca723c7547b48cf1ed7d5bc0cae5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:56:38.901155 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-06feaf37-ae52-47f0-b63c-904478de6770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:38.974078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1330/2668] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:56:38 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:56:38.986363 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3947ad-53a5-4b7d-83b2-833a9ad52d26 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:38.994293 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-1c3947ad-53a5-4b7d-83b2-833a9ad52d26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:39.039357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:56:39.055055 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3947ad-53a5-4b7d-83b2-833a9ad52d26 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:39.058706 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b8684a-7351-4585-ba20-87701fb2376e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:39.059372 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8684a-7351-4585-ba20-87701fb2376e service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:39.059653 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8684a-7351-4585-ba20-87701fb2376e service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:39.060482 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8684a-7351-4585-ba20-87701fb2376e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:39.122303 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b8684a-7351-4585-ba20-87701fb2376e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:39.124478 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b8684a-7351-4585-ba20-87701fb2376e service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:39.134484 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1339/2669] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:56:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:56:40.649681 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06feaf37-ae52-47f0-b63c-904478de6770 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:40.651217 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-06feaf37-ae52-47f0-b63c-904478de6770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:40.663995 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06feaf37-ae52-47f0-b63c-904478de6770 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:40.668193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407e39f7-4fed-499d-9843-6d809130ce1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:40.668834 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e39f7-4fed-499d-9843-6d809130ce1c service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:40.669148 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e39f7-4fed-499d-9843-6d809130ce1c service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:40.669591 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e39f7-4fed-499d-9843-6d809130ce1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:40.729705 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407e39f7-4fed-499d-9843-6d809130ce1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:40.731275 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407e39f7-4fed-499d-9843-6d809130ce1c service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:40.742604 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1331/2670] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:56:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:56:41.301165 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b8684a-7351-4585-ba20-87701fb2376e service cinder] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:41.311609 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b8684a-7351-4585-ba20-87701fb2376e service cinder] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:41.315265 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda10bd-36a2-42fb-9fb3-e539e541055f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:41.315903 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda10bd-36a2-42fb-9fb3-e539e541055f service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:41.316189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda10bd-36a2-42fb-9fb3-e539e541055f service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:41.316465 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda10bd-36a2-42fb-9fb3-e539e541055f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:41.325390 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fda10bd-36a2-42fb-9fb3-e539e541055f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:41.327503 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fda10bd-36a2-42fb-9fb3-e539e541055f service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:41.336915 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1340/2671] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:56:41 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:56:41.861881 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407e39f7-4fed-499d-9843-6d809130ce1c service cinder] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:41.863314 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-407e39f7-4fed-499d-9843-6d809130ce1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:41.897211 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:56:41.929774 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407e39f7-4fed-499d-9843-6d809130ce1c service cinder] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:41.934400 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb8ff54-ca1a-425b-9cb1-ab735ad9f37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:41.935272 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8ff54-ca1a-425b-9cb1-ab735ad9f37c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:41.935559 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8ff54-ca1a-425b-9cb1-ab735ad9f37c admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:41.935880 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8ff54-ca1a-425b-9cb1-ab735ad9f37c admin admin] PATH_INFO: `/v3/users/61dda783e9a74513819444fb64078cfb` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:41.937089 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb8ff54-ca1a-425b-9cb1-ab735ad9f37c admin admin] RBAC: Authorizing `identity:delete_user(user_id=61dda783e9a74513819444fb64078cfb)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:41.939054 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb8ff54-ca1a-425b-9cb1-ab735ad9f37c admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:41.986301 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-dbb8ff54-ca1a-425b-9cb1-ab735ad9f37c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61dda783e9a74513819444fb64078cfb'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:41.993294 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-dbb8ff54-ca1a-425b-9cb1-ab735ad9f37c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61dda783e9a74513819444fb64078cfb'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:41.993908 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-dbb8ff54-ca1a-425b-9cb1-ab735ad9f37c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61dda783e9a74513819444fb64078cfb'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:42.001217 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-dbb8ff54-ca1a-425b-9cb1-ab735ad9f37c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61dda783e9a74513819444fb64078cfb'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:42.012875 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-dbb8ff54-ca1a-425b-9cb1-ab735ad9f37c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61dda783e9a74513819444fb64078cfb'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:42.019053 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1332/2672] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:56:41 2022] DELETE /identity/v3/users/61dda783e9a74513819444fb64078cfb => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:56:42.023689 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047976e1-7094-4991-bc12-88ff089cc2c6 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:42.023993 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047976e1-7094-4991-bc12-88ff089cc2c6 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:42.024278 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047976e1-7094-4991-bc12-88ff089cc2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:42.041950 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-047976e1-7094-4991-bc12-88ff089cc2c6 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:56:42.043142 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-047976e1-7094-4991-bc12-88ff089cc2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:42.103271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1341/2673] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:56:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:56:42.114048 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb8ff54-ca1a-425b-9cb1-ab735ad9f37c admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:42.115551 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-dbb8ff54-ca1a-425b-9cb1-ab735ad9f37c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:42.173383 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb8ff54-ca1a-425b-9cb1-ab735ad9f37c admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:42.177283 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f46454-714b-4f4d-a383-3e6d11a9799d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:42.177927 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f46454-714b-4f4d-a383-3e6d11a9799d service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:42.178212 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f46454-714b-4f4d-a383-3e6d11a9799d service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:42.178487 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f46454-714b-4f4d-a383-3e6d11a9799d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:42.187572 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f46454-714b-4f4d-a383-3e6d11a9799d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:42.189009 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f46454-714b-4f4d-a383-3e6d11a9799d service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:42.198424 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1333/2674] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:56:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:56:42.465331 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047976e1-7094-4991-bc12-88ff089cc2c6 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:42.468235 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-047976e1-7094-4991-bc12-88ff089cc2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:42.481470 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047976e1-7094-4991-bc12-88ff089cc2c6 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:42.486443 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7bdd29-bf8f-48c8-8405-ccb2f9c13a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:42.487309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7bdd29-bf8f-48c8-8405-ccb2f9c13a6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:42.487588 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7bdd29-bf8f-48c8-8405-ccb2f9c13a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:42.487867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7bdd29-bf8f-48c8-8405-ccb2f9c13a6e admin admin] PATH_INFO: `/v3/projects/52dc411d6a7144889b45097422024c70` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:42.489064 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7bdd29-bf8f-48c8-8405-ccb2f9c13a6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=52dc411d6a7144889b45097422024c70)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:42.490949 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7bdd29-bf8f-48c8-8405-ccb2f9c13a6e admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:42.541197 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-cf7bdd29-bf8f-48c8-8405-ccb2f9c13a6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52dc411d6a7144889b45097422024c70', 'request_id': 'req-cf7bdd29-bf8f-48c8-8405-ccb2f9c13a6e'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:42.547197 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-cf7bdd29-bf8f-48c8-8405-ccb2f9c13a6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52dc411d6a7144889b45097422024c70', 'request_id': 'req-cf7bdd29-bf8f-48c8-8405-ccb2f9c13a6e'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:42.555143 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-cf7bdd29-bf8f-48c8-8405-ccb2f9c13a6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52dc411d6a7144889b45097422024c70', 'request_id': 'req-cf7bdd29-bf8f-48c8-8405-ccb2f9c13a6e'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:42.566338 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-cf7bdd29-bf8f-48c8-8405-ccb2f9c13a6e admin admin] The token cache is being invalidate because project 52dc411d6a7144889b45097422024c70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:56:42.566630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-cf7bdd29-bf8f-48c8-8405-ccb2f9c13a6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:42.567268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-cf7bdd29-bf8f-48c8-8405-ccb2f9c13a6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:42.568809 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1342/2675] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:56:42 2022] DELETE /identity/v3/projects/52dc411d6a7144889b45097422024c70 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:56:42.573956 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427f009-ff16-4bdb-9e41-1d841c3c2e18 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:42.574250 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427f009-ff16-4bdb-9e41-1d841c3c2e18 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:42.574532 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427f009-ff16-4bdb-9e41-1d841c3c2e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:42.595172 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-6427f009-ff16-4bdb-9e41-1d841c3c2e18 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:56:42.596538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-6427f009-ff16-4bdb-9e41-1d841c3c2e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:42.646431 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:56:42.668535 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1334/2676] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:56:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:56:42.670165 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c520646-b222-43df-8694-0193b25c435c None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:42.671236 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c520646-b222-43df-8694-0193b25c435c None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:42.671486 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c520646-b222-43df-8694-0193b25c435c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:42.679443 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:56:42.695684 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-4c520646-b222-43df-8694-0193b25c435c None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:56:42.697957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-4c520646-b222-43df-8694-0193b25c435c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:42.706929 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1343/2677] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:56:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:56:42.711218 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6427f009-ff16-4bdb-9e41-1d841c3c2e18 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:42.724290 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6427f009-ff16-4bdb-9e41-1d841c3c2e18 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:42.729309 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea811878-9419-48ff-a3d5-832549e86edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:42.730052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea811878-9419-48ff-a3d5-832549e86edd admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:42.730407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea811878-9419-48ff-a3d5-832549e86edd admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:42.730874 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea811878-9419-48ff-a3d5-832549e86edd admin admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:42.731822 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea811878-9419-48ff-a3d5-832549e86edd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:56:42.732314 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea811878-9419-48ff-a3d5-832549e86edd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:42.734259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea811878-9419-48ff-a3d5-832549e86edd admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:42.746251 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1335/2678] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:56:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:56:42.751244 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c520646-b222-43df-8694-0193b25c435c None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:42.761222 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c520646-b222-43df-8694-0193b25c435c None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:42.767119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad124e-190a-4742-8fc1-7872043a53e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:42.767789 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad124e-190a-4742-8fc1-7872043a53e6 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:42.768073 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad124e-190a-4742-8fc1-7872043a53e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:42.768351 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad124e-190a-4742-8fc1-7872043a53e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:42.769083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad124e-190a-4742-8fc1-7872043a53e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:42.770980 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad124e-190a-4742-8fc1-7872043a53e6 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:42.801818 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1344/2679] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:56:42 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:56:42.805821 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea811878-9419-48ff-a3d5-832549e86edd admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:42.816169 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea811878-9419-48ff-a3d5-832549e86edd admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:42.890419 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b6d5c5-58d6-49d6-943c-0b045923335b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:42.891139 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b6d5c5-58d6-49d6-943c-0b045923335b admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:42.891384 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b6d5c5-58d6-49d6-943c-0b045923335b admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:42.891670 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b6d5c5-58d6-49d6-943c-0b045923335b admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:42.892364 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b6d5c5-58d6-49d6-943c-0b045923335b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:42.894460 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b6d5c5-58d6-49d6-943c-0b045923335b admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:42.923451 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1336/2680] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:56:42 2022] POST /identity/v3/users => generated 287 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:56:42.924841 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad124e-190a-4742-8fc1-7872043a53e6 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:42.935447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad124e-190a-4742-8fc1-7872043a53e6 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:42.941110 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0016e-8b73-4e7f-986a-1d62e38af888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:42.941771 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0016e-8b73-4e7f-986a-1d62e38af888 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:42.942055 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0016e-8b73-4e7f-986a-1d62e38af888 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:42.942333 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0016e-8b73-4e7f-986a-1d62e38af888 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:42.943058 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a0016e-8b73-4e7f-986a-1d62e38af888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:42.944632 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a0016e-8b73-4e7f-986a-1d62e38af888 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:42.955710 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1345/2681] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:56:42.959782 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b6d5c5-58d6-49d6-943c-0b045923335b admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:42.969850 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b6d5c5-58d6-49d6-943c-0b045923335b admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:42.974405 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc8148-c8fd-4942-bf0a-7a9bc63c9752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:42.975116 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc8148-c8fd-4942-bf0a-7a9bc63c9752 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:42.975405 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc8148-c8fd-4942-bf0a-7a9bc63c9752 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:42.975684 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc8148-c8fd-4942-bf0a-7a9bc63c9752 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:42.976402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc8148-c8fd-4942-bf0a-7a9bc63c9752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:42.978032 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc8148-c8fd-4942-bf0a-7a9bc63c9752 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:42.988364 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1337/2682] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:56:42.992280 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0016e-8b73-4e7f-986a-1d62e38af888 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:43.002927 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0016e-8b73-4e7f-986a-1d62e38af888 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:43.007507 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ccde12-c859-473b-8233-8c279c7f9dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:43.008372 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ccde12-c859-473b-8233-8c279c7f9dfa admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:43.008652 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ccde12-c859-473b-8233-8c279c7f9dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:43.008932 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ccde12-c859-473b-8233-8c279c7f9dfa admin admin] PATH_INFO: `/v3/projects/c8ca80cbead344739f19271930c14d3b/users/f44d7a12a15c4d54b32e0e6e045be419/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:43.023109 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ccde12-c859-473b-8233-8c279c7f9dfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8ca80cbead344739f19271930c14d3b, user_id=f44d7a12a15c4d54b32e0e6e045be419, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:43.025750 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ccde12-c859-473b-8233-8c279c7f9dfa admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:43.038463 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1346/2683] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:56:42 2022] PUT /identity/v3/projects/c8ca80cbead344739f19271930c14d3b/users/f44d7a12a15c4d54b32e0e6e045be419/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:56:43.049311 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ed43e-db8a-4348-93ec-f374883c4253 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:43.049638 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ed43e-db8a-4348-93ec-f374883c4253 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:43.049926 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ed43e-db8a-4348-93ec-f374883c4253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:43.077450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-276ed43e-db8a-4348-93ec-f374883c4253 None None] MFA Rules not processed for user `f44d7a12a15c4d54b32e0e6e045be419`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:56:43.078711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-276ed43e-db8a-4348-93ec-f374883c4253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:43.142296 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1338/2684] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:56:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:56:43.153435 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ccde12-c859-473b-8233-8c279c7f9dfa admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:43.155219 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-b7ccde12-c859-473b-8233-8c279c7f9dfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:43.213938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ccde12-c859-473b-8233-8c279c7f9dfa admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:43.217521 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e4d65-e2b9-4d2a-a259-4640773404af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:43.218371 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e4d65-e2b9-4d2a-a259-4640773404af service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:43.218661 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e4d65-e2b9-4d2a-a259-4640773404af service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:43.218940 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e4d65-e2b9-4d2a-a259-4640773404af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:43.228454 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3e4d65-e2b9-4d2a-a259-4640773404af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:43.230279 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3e4d65-e2b9-4d2a-a259-4640773404af service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:43.241591 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1347/2685] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:56:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:56:44.596123 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b40752-05b7-456c-a163-41751ea3a984 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:44.596655 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b40752-05b7-456c-a163-41751ea3a984 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:44.596783 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b40752-05b7-456c-a163-41751ea3a984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:44.615022 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-50b40752-05b7-456c-a163-41751ea3a984 None None] MFA Rules not processed for user `f44d7a12a15c4d54b32e0e6e045be419`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:56:44.616206 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-50b40752-05b7-456c-a163-41751ea3a984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:44.623999 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1339/2686] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:56:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:56:45.199927 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e4d65-e2b9-4d2a-a259-4640773404af service cinder] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:45.201441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-0e3e4d65-e2b9-4d2a-a259-4640773404af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:45.272213 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e4d65-e2b9-4d2a-a259-4640773404af service cinder] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:45.276880 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee5fc7-520a-4941-a875-704560d11357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:45.277799 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee5fc7-520a-4941-a875-704560d11357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:45.278086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee5fc7-520a-4941-a875-704560d11357 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:45.278383 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee5fc7-520a-4941-a875-704560d11357 admin admin] PATH_INFO: `/v3/users/a5bf18812aaf4f4083d8f3aa32a1f0a6` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:45.280125 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fee5fc7-520a-4941-a875-704560d11357 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5bf18812aaf4f4083d8f3aa32a1f0a6)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:45.282307 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fee5fc7-520a-4941-a875-704560d11357 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:45.348913 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:56:45.351490 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-6fee5fc7-520a-4941-a875-704560d11357 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5bf18812aaf4f4083d8f3aa32a1f0a6'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:45.361595 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-6fee5fc7-520a-4941-a875-704560d11357 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5bf18812aaf4f4083d8f3aa32a1f0a6'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:45.371143 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-6fee5fc7-520a-4941-a875-704560d11357 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5bf18812aaf4f4083d8f3aa32a1f0a6'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:45.372030 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-6fee5fc7-520a-4941-a875-704560d11357 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5bf18812aaf4f4083d8f3aa32a1f0a6'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:45.386463 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-6fee5fc7-520a-4941-a875-704560d11357 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5bf18812aaf4f4083d8f3aa32a1f0a6'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:45.393324 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1348/2687] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:56:45 2022] DELETE /identity/v3/users/a5bf18812aaf4f4083d8f3aa32a1f0a6 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:56:45.398104 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b2eec-112f-42c3-8c9b-ccdc609963b5 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:45.398649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b2eec-112f-42c3-8c9b-ccdc609963b5 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:45.398933 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b2eec-112f-42c3-8c9b-ccdc609963b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:45.417787 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-cc4b2eec-112f-42c3-8c9b-ccdc609963b5 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:56:45.418930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-cc4b2eec-112f-42c3-8c9b-ccdc609963b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:45.465236 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:56:45.491895 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1340/2688] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:56:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:56:45.499980 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee5fc7-520a-4941-a875-704560d11357 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:45.501570 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-6fee5fc7-520a-4941-a875-704560d11357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:45.566764 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee5fc7-520a-4941-a875-704560d11357 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:45.571845 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643a9e4-2d49-4e9b-b79d-aecec5c15a5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:45.572532 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643a9e4-2d49-4e9b-b79d-aecec5c15a5f service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:45.572813 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643a9e4-2d49-4e9b-b79d-aecec5c15a5f service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:45.573122 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643a9e4-2d49-4e9b-b79d-aecec5c15a5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:45.585424 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e643a9e4-2d49-4e9b-b79d-aecec5c15a5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:45.586848 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e643a9e4-2d49-4e9b-b79d-aecec5c15a5f service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:45.609083 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1349/2689] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:56:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:56:45.888620 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b2eec-112f-42c3-8c9b-ccdc609963b5 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:45.890804 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-cc4b2eec-112f-42c3-8c9b-ccdc609963b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:45.904121 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b2eec-112f-42c3-8c9b-ccdc609963b5 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:45.909427 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1040d2ee-b499-4bc5-bd62-b1437b27b153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:45.910312 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040d2ee-b499-4bc5-bd62-b1437b27b153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:45.910594 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040d2ee-b499-4bc5-bd62-b1437b27b153 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:45.911063 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040d2ee-b499-4bc5-bd62-b1437b27b153 admin admin] PATH_INFO: `/v3/projects/5486bfe046db4a7e883a6307e0c0f61c` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:45.912263 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1040d2ee-b499-4bc5-bd62-b1437b27b153 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5486bfe046db4a7e883a6307e0c0f61c)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:45.914167 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1040d2ee-b499-4bc5-bd62-b1437b27b153 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:45.960297 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-1040d2ee-b499-4bc5-bd62-b1437b27b153 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5486bfe046db4a7e883a6307e0c0f61c', 'request_id': 'req-1040d2ee-b499-4bc5-bd62-b1437b27b153'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:45.966229 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-1040d2ee-b499-4bc5-bd62-b1437b27b153 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5486bfe046db4a7e883a6307e0c0f61c', 'request_id': 'req-1040d2ee-b499-4bc5-bd62-b1437b27b153'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:45.974686 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-1040d2ee-b499-4bc5-bd62-b1437b27b153 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5486bfe046db4a7e883a6307e0c0f61c', 'request_id': 'req-1040d2ee-b499-4bc5-bd62-b1437b27b153'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:45.985797 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-1040d2ee-b499-4bc5-bd62-b1437b27b153 admin admin] The token cache is being invalidate because project 5486bfe046db4a7e883a6307e0c0f61c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:56:45.986196 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-1040d2ee-b499-4bc5-bd62-b1437b27b153 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:45.986833 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-1040d2ee-b499-4bc5-bd62-b1437b27b153 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:45.988385 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1341/2690] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:56:45 2022] DELETE /identity/v3/projects/5486bfe046db4a7e883a6307e0c0f61c => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:56:45.992377 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643a9e4-2d49-4e9b-b79d-aecec5c15a5f service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:46.001973 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643a9e4-2d49-4e9b-b79d-aecec5c15a5f service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:46.059601 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4499da-55e0-4b76-8ef5-867d8eb5d36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:46.060471 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4499da-55e0-4b76-8ef5-867d8eb5d36c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:46.060778 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4499da-55e0-4b76-8ef5-867d8eb5d36c admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:46.061297 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4499da-55e0-4b76-8ef5-867d8eb5d36c admin admin] PATH_INFO: `/v3/users/c87d1d2c23be4a38a410ddc72d27ebcb` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:46.062485 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4499da-55e0-4b76-8ef5-867d8eb5d36c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c87d1d2c23be4a38a410ddc72d27ebcb)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:46.064379 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4499da-55e0-4b76-8ef5-867d8eb5d36c admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:46.112518 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-bd4499da-55e0-4b76-8ef5-867d8eb5d36c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c87d1d2c23be4a38a410ddc72d27ebcb'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:46.121174 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-bd4499da-55e0-4b76-8ef5-867d8eb5d36c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c87d1d2c23be4a38a410ddc72d27ebcb'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:46.129574 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-bd4499da-55e0-4b76-8ef5-867d8eb5d36c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c87d1d2c23be4a38a410ddc72d27ebcb'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:46.130221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-bd4499da-55e0-4b76-8ef5-867d8eb5d36c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c87d1d2c23be4a38a410ddc72d27ebcb'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:46.135234 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:56:46.145373 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-bd4499da-55e0-4b76-8ef5-867d8eb5d36c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c87d1d2c23be4a38a410ddc72d27ebcb'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:46.153854 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1350/2691] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:56:45 2022] DELETE /identity/v3/users/c87d1d2c23be4a38a410ddc72d27ebcb => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:56:46.514937 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1040d2ee-b499-4bc5-bd62-b1437b27b153 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:46.516442 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-1040d2ee-b499-4bc5-bd62-b1437b27b153 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:46.534587 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1040d2ee-b499-4bc5-bd62-b1437b27b153 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:46.610317 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff8da2-9407-4ca0-a621-727bae7c45d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:46.611295 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff8da2-9407-4ca0-a621-727bae7c45d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:46.611580 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff8da2-9407-4ca0-a621-727bae7c45d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:46.612064 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff8da2-9407-4ca0-a621-727bae7c45d0 admin admin] PATH_INFO: `/v3/projects/ffae6d693de743fe96d94283d2e22ba6` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:46.613659 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff8da2-9407-4ca0-a621-727bae7c45d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffae6d693de743fe96d94283d2e22ba6)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:46.615813 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff8da2-9407-4ca0-a621-727bae7c45d0 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:46.672683 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-81ff8da2-9407-4ca0-a621-727bae7c45d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffae6d693de743fe96d94283d2e22ba6', 'request_id': 'req-81ff8da2-9407-4ca0-a621-727bae7c45d0'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:46.679587 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-81ff8da2-9407-4ca0-a621-727bae7c45d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffae6d693de743fe96d94283d2e22ba6', 'request_id': 'req-81ff8da2-9407-4ca0-a621-727bae7c45d0'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:46.687944 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-81ff8da2-9407-4ca0-a621-727bae7c45d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffae6d693de743fe96d94283d2e22ba6', 'request_id': 'req-81ff8da2-9407-4ca0-a621-727bae7c45d0'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:46.692965 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:56:46.700225 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-81ff8da2-9407-4ca0-a621-727bae7c45d0 admin admin] The token cache is being invalidate because project ffae6d693de743fe96d94283d2e22ba6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:56:46.700701 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-81ff8da2-9407-4ca0-a621-727bae7c45d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:46.702582 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-81ff8da2-9407-4ca0-a621-727bae7c45d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:46.704378 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1342/2692] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:56:46 2022] DELETE /identity/v3/projects/ffae6d693de743fe96d94283d2e22ba6 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:56:46.708268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4499da-55e0-4b76-8ef5-867d8eb5d36c admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:46.709786 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-bd4499da-55e0-4b76-8ef5-867d8eb5d36c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:46.722624 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4499da-55e0-4b76-8ef5-867d8eb5d36c admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:46.784537 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a34e6-6fc3-4870-9d08-355fac547abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:46.785458 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a34e6-6fc3-4870-9d08-355fac547abb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:46.785745 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a34e6-6fc3-4870-9d08-355fac547abb admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:46.786214 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a34e6-6fc3-4870-9d08-355fac547abb admin admin] PATH_INFO: `/v3/users/d73eca723c7547b48cf1ed7d5bc0cae5` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:46.787421 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062a34e6-6fc3-4870-9d08-355fac547abb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d73eca723c7547b48cf1ed7d5bc0cae5)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:46.789425 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062a34e6-6fc3-4870-9d08-355fac547abb admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:46.839167 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-062a34e6-6fc3-4870-9d08-355fac547abb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd73eca723c7547b48cf1ed7d5bc0cae5'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:46.847317 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-062a34e6-6fc3-4870-9d08-355fac547abb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd73eca723c7547b48cf1ed7d5bc0cae5'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:46.855134 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-062a34e6-6fc3-4870-9d08-355fac547abb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd73eca723c7547b48cf1ed7d5bc0cae5'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:46.855845 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-062a34e6-6fc3-4870-9d08-355fac547abb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd73eca723c7547b48cf1ed7d5bc0cae5'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:46.872618 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-062a34e6-6fc3-4870-9d08-355fac547abb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd73eca723c7547b48cf1ed7d5bc0cae5'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:46.881065 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1351/2693] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:56:46 2022] DELETE /identity/v3/users/d73eca723c7547b48cf1ed7d5bc0cae5 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:56:47.163395 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff8da2-9407-4ca0-a621-727bae7c45d0 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:47.165461 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-81ff8da2-9407-4ca0-a621-727bae7c45d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:47.180878 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff8da2-9407-4ca0-a621-727bae7c45d0 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:47.267200 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1daaae-2b40-4b3b-b539-013198d8415d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:47.268089 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1daaae-2b40-4b3b-b539-013198d8415d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:47.268370 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1daaae-2b40-4b3b-b539-013198d8415d admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:47.269325 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1daaae-2b40-4b3b-b539-013198d8415d admin admin] PATH_INFO: `/v3/projects/9a0c8ea7168f40c7bb988fb895704c14` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:47.270775 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1daaae-2b40-4b3b-b539-013198d8415d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a0c8ea7168f40c7bb988fb895704c14)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:47.272930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1daaae-2b40-4b3b-b539-013198d8415d admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:47.344519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-6b1daaae-2b40-4b3b-b539-013198d8415d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a0c8ea7168f40c7bb988fb895704c14', 'request_id': 'req-6b1daaae-2b40-4b3b-b539-013198d8415d'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:47.352486 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-6b1daaae-2b40-4b3b-b539-013198d8415d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a0c8ea7168f40c7bb988fb895704c14', 'request_id': 'req-6b1daaae-2b40-4b3b-b539-013198d8415d'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:47.363829 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-6b1daaae-2b40-4b3b-b539-013198d8415d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a0c8ea7168f40c7bb988fb895704c14', 'request_id': 'req-6b1daaae-2b40-4b3b-b539-013198d8415d'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:47.379515 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-6b1daaae-2b40-4b3b-b539-013198d8415d admin admin] The token cache is being invalidate because project 9a0c8ea7168f40c7bb988fb895704c14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:56:47.379515 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-6b1daaae-2b40-4b3b-b539-013198d8415d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:47.380405 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-6b1daaae-2b40-4b3b-b539-013198d8415d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:56:47.382383 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1343/2694] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:56:47 2022] DELETE /identity/v3/projects/9a0c8ea7168f40c7bb988fb895704c14 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:56:47.387878 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f412dc-44bb-4988-b0e1-45e3b3be2258 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:47.388169 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f412dc-44bb-4988-b0e1-45e3b3be2258 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:47.388456 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f412dc-44bb-4988-b0e1-45e3b3be2258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:47.413859 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-68f412dc-44bb-4988-b0e1-45e3b3be2258 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:56:47.415121 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-68f412dc-44bb-4988-b0e1-45e3b3be2258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:47.497041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1352/2695] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:56:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:56:47.503598 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8782421-d014-4636-897e-e3634e776db1 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:47.503893 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8782421-d014-4636-897e-e3634e776db1 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:47.504599 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8782421-d014-4636-897e-e3634e776db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:47.524498 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-a8782421-d014-4636-897e-e3634e776db1 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:56:47.526013 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a8782421-d014-4636-897e-e3634e776db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:47.537936 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1344/2696] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:56:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:56:47.538217 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f412dc-44bb-4988-b0e1-45e3b3be2258 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:47.544591 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:56:47.551343 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f412dc-44bb-4988-b0e1-45e3b3be2258 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:47.566337 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfda53d-3a42-443d-b8d6-a56387b8fcf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:47.567122 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfda53d-3a42-443d-b8d6-a56387b8fcf5 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:47.567517 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfda53d-3a42-443d-b8d6-a56387b8fcf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:47.567865 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfda53d-3a42-443d-b8d6-a56387b8fcf5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:47.568925 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfda53d-3a42-443d-b8d6-a56387b8fcf5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:56:47.569403 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfda53d-3a42-443d-b8d6-a56387b8fcf5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:47.571302 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfda53d-3a42-443d-b8d6-a56387b8fcf5 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:47.584488 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1353/2697] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:56:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 13:56:47.588899 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8782421-d014-4636-897e-e3634e776db1 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:47.600124 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8782421-d014-4636-897e-e3634e776db1 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:47.605634 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9342f77-e3b8-4ab8-a2dd-747367a3724d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:47.606315 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9342f77-e3b8-4ab8-a2dd-747367a3724d admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:47.606624 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9342f77-e3b8-4ab8-a2dd-747367a3724d admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:47.606889 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9342f77-e3b8-4ab8-a2dd-747367a3724d admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:47.608262 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9342f77-e3b8-4ab8-a2dd-747367a3724d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:47.610608 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9342f77-e3b8-4ab8-a2dd-747367a3724d admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:47.650326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1345/2698] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:56:47 2022] POST /identity/v3/projects => generated 363 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:56:47.654050 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfda53d-3a42-443d-b8d6-a56387b8fcf5 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:47.667163 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfda53d-3a42-443d-b8d6-a56387b8fcf5 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:47.708159 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9342f77-e3b8-4ab8-a2dd-747367a3724d admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:47.710134 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a9342f77-e3b8-4ab8-a2dd-747367a3724d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:47.787842 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ffec4f-3ae4-44fb-8d91-1282d376524c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:47.788663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ffec4f-3ae4-44fb-8d91-1282d376524c admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:47.788958 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ffec4f-3ae4-44fb-8d91-1282d376524c admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:47.789340 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ffec4f-3ae4-44fb-8d91-1282d376524c admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:47.792180 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ffec4f-3ae4-44fb-8d91-1282d376524c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:47.797896 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ffec4f-3ae4-44fb-8d91-1282d376524c admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:47.819922 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:56:47.837882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1354/2699] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:56:47 2022] POST /identity/v3/users => generated 282 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:56:47.841451 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ffec4f-3ae4-44fb-8d91-1282d376524c admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:47.852427 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ffec4f-3ae4-44fb-8d91-1282d376524c admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:47.854138 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9342f77-e3b8-4ab8-a2dd-747367a3724d admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:47.858239 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b6120-2d8f-43b4-91ba-bc0e3eba918b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:47.858939 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b6120-2d8f-43b4-91ba-bc0e3eba918b admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:47.859296 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b6120-2d8f-43b4-91ba-bc0e3eba918b admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:47.859703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b6120-2d8f-43b4-91ba-bc0e3eba918b admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:47.860444 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b6120-2d8f-43b4-91ba-bc0e3eba918b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:47.861330 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17422fae-8772-4e72-a3ca-e2d3a9b2cbf6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:47.862085 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b6120-2d8f-43b4-91ba-bc0e3eba918b admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:47.862932 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17422fae-8772-4e72-a3ca-e2d3a9b2cbf6 service nova] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:47.863287 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17422fae-8772-4e72-a3ca-e2d3a9b2cbf6 service nova] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:47.863873 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17422fae-8772-4e72-a3ca-e2d3a9b2cbf6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:47.866444 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-17422fae-8772-4e72-a3ca-e2d3a9b2cbf6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:47.874252 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1355/2700] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:56:47.879127 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b6120-2d8f-43b4-91ba-bc0e3eba918b admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:47.883048 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17422fae-8772-4e72-a3ca-e2d3a9b2cbf6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:47.884699 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17422fae-8772-4e72-a3ca-e2d3a9b2cbf6 service nova] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:47.891358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b6120-2d8f-43b4-91ba-bc0e3eba918b admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:47.898015 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1346/2701] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:56:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:56:47.909326 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18ff556-c9c9-4d1a-94db-55a69b2ef3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:47.910021 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ff556-c9c9-4d1a-94db-55a69b2ef3bd admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:47.910298 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ff556-c9c9-4d1a-94db-55a69b2ef3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:47.910569 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ff556-c9c9-4d1a-94db-55a69b2ef3bd admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:47.911437 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18ff556-c9c9-4d1a-94db-55a69b2ef3bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:47.914352 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18ff556-c9c9-4d1a-94db-55a69b2ef3bd admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:47.934834 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1356/2702] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:56:47.938904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17422fae-8772-4e72-a3ca-e2d3a9b2cbf6 service nova] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:47.956017 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17422fae-8772-4e72-a3ca-e2d3a9b2cbf6 service nova] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:47.962860 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa30d24-96be-4d15-9b28-31d3d17de88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:47.963785 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa30d24-96be-4d15-9b28-31d3d17de88e admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:47.964076 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa30d24-96be-4d15-9b28-31d3d17de88e admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:47.964352 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa30d24-96be-4d15-9b28-31d3d17de88e admin admin] PATH_INFO: `/v3/projects/880a0217d7654bbaaf448404cca1b62c/users/0b4c1257a02d471d84a5a04cc027f992/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:47.982408 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa30d24-96be-4d15-9b28-31d3d17de88e admin admin] RBAC: Authorizing `identity:create_grant(project_id=880a0217d7654bbaaf448404cca1b62c, user_id=0b4c1257a02d471d84a5a04cc027f992, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:47.985720 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa30d24-96be-4d15-9b28-31d3d17de88e admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:48.007500 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1347/2703] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:56:47 2022] PUT /identity/v3/projects/880a0217d7654bbaaf448404cca1b62c/users/0b4c1257a02d471d84a5a04cc027f992/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:56:48.018041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc733ad-3c13-467b-afb4-0463541ae86a None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:48.018511 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc733ad-3c13-467b-afb4-0463541ae86a None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:48.018915 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc733ad-3c13-467b-afb4-0463541ae86a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:48.050306 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-7bc733ad-3c13-467b-afb4-0463541ae86a None None] MFA Rules not processed for user `0b4c1257a02d471d84a5a04cc027f992`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:56:48.052452 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-7bc733ad-3c13-467b-afb4-0463541ae86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:48.171275 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1357/2704] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:56:48 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 13:56:48.180588 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa30d24-96be-4d15-9b28-31d3d17de88e admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:48.182567 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-0aa30d24-96be-4d15-9b28-31d3d17de88e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:48.256452 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa30d24-96be-4d15-9b28-31d3d17de88e admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:48.260947 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3adb9-33c1-4a09-bbb6-b5ebfb3323e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:48.261793 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3adb9-33c1-4a09-bbb6-b5ebfb3323e5 service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:48.262086 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3adb9-33c1-4a09-bbb6-b5ebfb3323e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:48.262369 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3adb9-33c1-4a09-bbb6-b5ebfb3323e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:48.272478 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba3adb9-33c1-4a09-bbb6-b5ebfb3323e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:48.274124 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba3adb9-33c1-4a09-bbb6-b5ebfb3323e5 service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:48.290349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1348/2705] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:56:48 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:56:52.737047 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc733ad-3c13-467b-afb4-0463541ae86a None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:52.738648 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-7bc733ad-3c13-467b-afb4-0463541ae86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:52.798789 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:56:52.803342 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc733ad-3c13-467b-afb4-0463541ae86a None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:52.807854 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c5d2c-43c9-48a6-bef8-26959c457c41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:52.808512 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c5d2c-43c9-48a6-bef8-26959c457c41 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:52.808874 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c5d2c-43c9-48a6-bef8-26959c457c41 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:52.809197 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c5d2c-43c9-48a6-bef8-26959c457c41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:52.811018 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-b31c5d2c-43c9-48a6-bef8-26959c457c41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:52.876078 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c5d2c-43c9-48a6-bef8-26959c457c41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:52.877551 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c5d2c-43c9-48a6-bef8-26959c457c41 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:52.887251 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1358/2706] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:56:52 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:56:54.606709 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3adb9-33c1-4a09-bbb6-b5ebfb3323e5 service cinder] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:56:54.608438 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-9ba3adb9-33c1-4a09-bbb6-b5ebfb3323e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:54.696194 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3adb9-33c1-4a09-bbb6-b5ebfb3323e5 service cinder] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:56:54.703320 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe20042-de34-42c7-9880-5536f0f1cbab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:56:54.704075 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe20042-de34-42c7-9880-5536f0f1cbab service nova] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:56:54.704803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe20042-de34-42c7-9880-5536f0f1cbab service nova] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:56:54.704803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe20042-de34-42c7-9880-5536f0f1cbab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:56:54.707451 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-efe20042-de34-42c7-9880-5536f0f1cbab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:56:54.721469 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe20042-de34-42c7-9880-5536f0f1cbab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:56:54.723213 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe20042-de34-42c7-9880-5536f0f1cbab service nova] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:56:54.736396 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1349/2707] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:56:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:57:02.882790 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=28 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:57:04.731433 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=78,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:57:05.604115 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270ff8d-4ab1-4c03-b316-c7187d0f0b9e None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:57:05.604612 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270ff8d-4ab1-4c03-b316-c7187d0f0b9e None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:57:05.604898 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270ff8d-4ab1-4c03-b316-c7187d0f0b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:57:05.622220 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-6270ff8d-4ab1-4c03-b316-c7187d0f0b9e None None] MFA Rules not processed for user `c4535134181e489fb532f617264d5855`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:57:05.623442 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-6270ff8d-4ab1-4c03-b316-c7187d0f0b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:57:05.683716 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1359/2708] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Jul 6 13:57:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:57:05.696632 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe20042-de34-42c7-9880-5536f0f1cbab service nova] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:57:05.698125 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-efe20042-de34-42c7-9880-5536f0f1cbab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:57:05.792432 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe20042-de34-42c7-9880-5536f0f1cbab service nova] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:57:05.797613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089f9e3-6598-47d2-9343-0dc4b3e743ca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c1bea05c37646b0a05fa8dda0367ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:57:05.798266 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089f9e3-6598-47d2-9343-0dc4b3e743ca service swift] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:57:05.798552 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089f9e3-6598-47d2-9343-0dc4b3e743ca service swift] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:57:05.798819 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089f9e3-6598-47d2-9343-0dc4b3e743ca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:57:05.808826 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b089f9e3-6598-47d2-9343-0dc4b3e743ca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:57:05.811375 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b089f9e3-6598-47d2-9343-0dc4b3e743ca service swift] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:57:05.819491 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1350/2709] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jul 6 13:57:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 06 13:57:06.338843 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07223c77-c6e6-49ba-8b82-0239c17dd330 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:57:06.339136 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07223c77-c6e6-49ba-8b82-0239c17dd330 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:57:06.339416 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07223c77-c6e6-49ba-8b82-0239c17dd330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:57:06.356910 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-07223c77-c6e6-49ba-8b82-0239c17dd330 None None] MFA Rules not processed for user `c4535134181e489fb532f617264d5855`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:57:06.358741 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-07223c77-c6e6-49ba-8b82-0239c17dd330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:57:06.367312 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1360/2710] 10.222.0.20 () {58 vars in 967 bytes} [Wed Jul 6 13:57:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:57:06.380040 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089f9e3-6598-47d2-9343-0dc4b3e743ca service swift] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:57:06.390784 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089f9e3-6598-47d2-9343-0dc4b3e743ca service swift] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:57:06.395948 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37752498-1fdf-4ea6-925e-f177aba95ada service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c1bea05c37646b0a05fa8dda0367ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:57:06.396595 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37752498-1fdf-4ea6-925e-f177aba95ada service swift] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:57:06.396875 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37752498-1fdf-4ea6-925e-f177aba95ada service swift] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:57:06.397180 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37752498-1fdf-4ea6-925e-f177aba95ada service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:57:06.405823 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37752498-1fdf-4ea6-925e-f177aba95ada service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:57:06.407397 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37752498-1fdf-4ea6-925e-f177aba95ada service swift] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:57:06.417154 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1351/2711] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jul 6 13:57:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 13:57:06.613754 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a9944-cf63-4253-a33b-182ae0e7edda None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:57:06.614247 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a9944-cf63-4253-a33b-182ae0e7edda None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:57:06.614352 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a9944-cf63-4253-a33b-182ae0e7edda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:57:06.633195 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-898a9944-cf63-4253-a33b-182ae0e7edda None None] MFA Rules not processed for user `c4535134181e489fb532f617264d5855`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:57:06.634391 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-898a9944-cf63-4253-a33b-182ae0e7edda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:57:06.641714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1361/2712] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Jul 6 13:57:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:57:06.653838 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37752498-1fdf-4ea6-925e-f177aba95ada service swift] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:57:06.665521 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37752498-1fdf-4ea6-925e-f177aba95ada service swift] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:57:06.671343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e7e80-9cf5-411e-a5d7-30ee4415426c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c1bea05c37646b0a05fa8dda0367ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:57:06.671975 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e7e80-9cf5-411e-a5d7-30ee4415426c service swift] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:57:06.672257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e7e80-9cf5-411e-a5d7-30ee4415426c service swift] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:57:06.672532 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e7e80-9cf5-411e-a5d7-30ee4415426c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:57:06.681066 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12e7e80-9cf5-411e-a5d7-30ee4415426c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:57:06.682566 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12e7e80-9cf5-411e-a5d7-30ee4415426c service swift] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:57:06.690989 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1352/2713] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jul 6 13:57:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 13:57:07.014338 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa37a9c-1c39-4703-b314-8f68150ed946 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:57:07.014588 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa37a9c-1c39-4703-b314-8f68150ed946 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:57:07.014871 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa37a9c-1c39-4703-b314-8f68150ed946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:57:07.033281 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-5aa37a9c-1c39-4703-b314-8f68150ed946 None None] MFA Rules not processed for user `c4535134181e489fb532f617264d5855`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:57:07.034411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-5aa37a9c-1c39-4703-b314-8f68150ed946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:57:07.041568 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1362/2714] 10.222.0.20 () {58 vars in 967 bytes} [Wed Jul 6 13:57:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:57:07.053970 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e7e80-9cf5-411e-a5d7-30ee4415426c service swift] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:57:07.066499 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e7e80-9cf5-411e-a5d7-30ee4415426c service swift] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:57:07.071550 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56313d89-c9ed-4010-81f4-4912ddcf2649 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c1bea05c37646b0a05fa8dda0367ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:57:07.072190 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56313d89-c9ed-4010-81f4-4912ddcf2649 service swift] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:57:07.072475 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56313d89-c9ed-4010-81f4-4912ddcf2649 service swift] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:57:07.072750 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56313d89-c9ed-4010-81f4-4912ddcf2649 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:57:07.082406 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56313d89-c9ed-4010-81f4-4912ddcf2649 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:57:07.084450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56313d89-c9ed-4010-81f4-4912ddcf2649 service swift] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:57:07.095873 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1353/2715] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jul 6 13:57:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 13:57:09.164830 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa37a9c-1c39-4703-b314-8f68150ed946 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:57:09.166255 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-5aa37a9c-1c39-4703-b314-8f68150ed946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:57:09.233809 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa37a9c-1c39-4703-b314-8f68150ed946 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:57:09.238885 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e87fe6-8109-474a-bb15-a5ed9d244812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:57:09.239746 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e87fe6-8109-474a-bb15-a5ed9d244812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:57:09.240026 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e87fe6-8109-474a-bb15-a5ed9d244812 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:57:09.240301 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e87fe6-8109-474a-bb15-a5ed9d244812 admin admin] PATH_INFO: `/v3/users/f44d7a12a15c4d54b32e0e6e045be419` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:57:09.241521 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e87fe6-8109-474a-bb15-a5ed9d244812 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f44d7a12a15c4d54b32e0e6e045be419)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:57:09.243407 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e87fe6-8109-474a-bb15-a5ed9d244812 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:57:09.302206 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-e3e87fe6-8109-474a-bb15-a5ed9d244812 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f44d7a12a15c4d54b32e0e6e045be419'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:57:09.310849 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-e3e87fe6-8109-474a-bb15-a5ed9d244812 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f44d7a12a15c4d54b32e0e6e045be419'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:57:09.318007 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-e3e87fe6-8109-474a-bb15-a5ed9d244812 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f44d7a12a15c4d54b32e0e6e045be419'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:57:09.318631 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-e3e87fe6-8109-474a-bb15-a5ed9d244812 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f44d7a12a15c4d54b32e0e6e045be419'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:57:09.330343 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-e3e87fe6-8109-474a-bb15-a5ed9d244812 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f44d7a12a15c4d54b32e0e6e045be419'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:57:09.336437 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1363/2716] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:57:09 2022] DELETE /identity/v3/users/f44d7a12a15c4d54b32e0e6e045be419 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:57:09.341108 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5c724-1bda-4c50-b55d-b5002545d7e0 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:57:09.341622 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5c724-1bda-4c50-b55d-b5002545d7e0 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:57:09.342065 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5c724-1bda-4c50-b55d-b5002545d7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:57:09.360023 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-9df5c724-1bda-4c50-b55d-b5002545d7e0 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:57:09.361230 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-9df5c724-1bda-4c50-b55d-b5002545d7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:57:09.420079 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1354/2717] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:57:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:57:09.430201 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e87fe6-8109-474a-bb15-a5ed9d244812 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:57:09.431731 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-e3e87fe6-8109-474a-bb15-a5ed9d244812 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:57:09.439006 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:57:09.492943 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e87fe6-8109-474a-bb15-a5ed9d244812 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:57:09.496633 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4d276-18e5-4415-b94c-58c94db04478 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:57:09.497342 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4d276-18e5-4415-b94c-58c94db04478 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:57:09.497640 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4d276-18e5-4415-b94c-58c94db04478 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:57:09.498492 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4d276-18e5-4415-b94c-58c94db04478 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:57:09.508980 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc4d276-18e5-4415-b94c-58c94db04478 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:57:09.510771 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc4d276-18e5-4415-b94c-58c94db04478 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:57:09.522485 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1364/2718] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:57:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:57:09.706773 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5c724-1bda-4c50-b55d-b5002545d7e0 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:57:09.708212 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-9df5c724-1bda-4c50-b55d-b5002545d7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:57:09.723617 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5c724-1bda-4c50-b55d-b5002545d7e0 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:57:09.728331 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c96e81-7af7-453f-bc6e-7f870e489de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:57:09.729228 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c96e81-7af7-453f-bc6e-7f870e489de2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:57:09.729514 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c96e81-7af7-453f-bc6e-7f870e489de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:57:09.729793 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c96e81-7af7-453f-bc6e-7f870e489de2 admin admin] PATH_INFO: `/v3/projects/c8ca80cbead344739f19271930c14d3b` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:57:09.731027 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c96e81-7af7-453f-bc6e-7f870e489de2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8ca80cbead344739f19271930c14d3b)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:57:09.733941 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c96e81-7af7-453f-bc6e-7f870e489de2 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:57:09.778715 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:57:09.782740 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-79c96e81-7af7-453f-bc6e-7f870e489de2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8ca80cbead344739f19271930c14d3b', 'request_id': 'req-79c96e81-7af7-453f-bc6e-7f870e489de2'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:57:09.789326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-79c96e81-7af7-453f-bc6e-7f870e489de2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8ca80cbead344739f19271930c14d3b', 'request_id': 'req-79c96e81-7af7-453f-bc6e-7f870e489de2'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:57:09.797276 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-79c96e81-7af7-453f-bc6e-7f870e489de2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8ca80cbead344739f19271930c14d3b', 'request_id': 'req-79c96e81-7af7-453f-bc6e-7f870e489de2'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:57:09.808344 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-79c96e81-7af7-453f-bc6e-7f870e489de2 admin admin] The token cache is being invalidate because project c8ca80cbead344739f19271930c14d3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:57:09.808635 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-79c96e81-7af7-453f-bc6e-7f870e489de2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:57:09.809312 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-79c96e81-7af7-453f-bc6e-7f870e489de2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:57:09.810901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1355/2719] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:57:09 2022] DELETE /identity/v3/projects/c8ca80cbead344739f19271930c14d3b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:57:09.815856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e4d93-ec15-4cca-97a6-5d348edf8226 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:57:09.816146 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e4d93-ec15-4cca-97a6-5d348edf8226 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:57:09.816443 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e4d93-ec15-4cca-97a6-5d348edf8226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:57:09.833938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-379e4d93-ec15-4cca-97a6-5d348edf8226 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:57:09.835140 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-379e4d93-ec15-4cca-97a6-5d348edf8226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:57:09.894718 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1365/2720] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:57:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:57:09.902466 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328f7d2-18d4-4782-92d1-1a424a53cde2 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:57:09.902766 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328f7d2-18d4-4782-92d1-1a424a53cde2 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:57:09.903047 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328f7d2-18d4-4782-92d1-1a424a53cde2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:57:09.922953 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-4328f7d2-18d4-4782-92d1-1a424a53cde2 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:57:09.924278 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-4328f7d2-18d4-4782-92d1-1a424a53cde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:57:09.933640 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1356/2721] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:57:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:57:09.937884 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e4d93-ec15-4cca-97a6-5d348edf8226 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:57:09.948581 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e4d93-ec15-4cca-97a6-5d348edf8226 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:57:09.953737 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2edc4f5-e6c1-4707-ab15-d40457ee60c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:57:09.954336 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2edc4f5-e6c1-4707-ab15-d40457ee60c5 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:57:09.954618 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2edc4f5-e6c1-4707-ab15-d40457ee60c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:57:09.954898 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2edc4f5-e6c1-4707-ab15-d40457ee60c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:57:09.955629 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2edc4f5-e6c1-4707-ab15-d40457ee60c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 13:57:09.955936 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2edc4f5-e6c1-4707-ab15-d40457ee60c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:57:09.957633 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2edc4f5-e6c1-4707-ab15-d40457ee60c5 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:57:09.973999 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1366/2722] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 13:57:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 13:57:09.975234 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4328f7d2-18d4-4782-92d1-1a424a53cde2 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:57:09.986038 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4328f7d2-18d4-4782-92d1-1a424a53cde2 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:57:09.990652 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20fa6fc-5222-4fc2-a9c6-b4296a2aeebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:57:09.991307 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20fa6fc-5222-4fc2-a9c6-b4296a2aeebc admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:57:09.991587 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20fa6fc-5222-4fc2-a9c6-b4296a2aeebc admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:57:09.991857 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20fa6fc-5222-4fc2-a9c6-b4296a2aeebc admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:57:09.992655 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20fa6fc-5222-4fc2-a9c6-b4296a2aeebc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:57:09.994574 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20fa6fc-5222-4fc2-a9c6-b4296a2aeebc admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:57:10.020994 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1357/2723] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 13:57:09 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 13:57:10.025372 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2edc4f5-e6c1-4707-ab15-d40457ee60c5 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:57:10.036314 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2edc4f5-e6c1-4707-ab15-d40457ee60c5 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:57:10.094022 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d28849f-1a5e-46bb-a653-858bf95a6f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:57:10.094704 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28849f-1a5e-46bb-a653-858bf95a6f25 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:57:10.094989 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28849f-1a5e-46bb-a653-858bf95a6f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:57:10.095286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28849f-1a5e-46bb-a653-858bf95a6f25 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:57:10.095980 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d28849f-1a5e-46bb-a653-858bf95a6f25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:57:10.097940 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d28849f-1a5e-46bb-a653-858bf95a6f25 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:57:10.118417 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1367/2724] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 13:57:10 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 13:57:10.123018 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20fa6fc-5222-4fc2-a9c6-b4296a2aeebc admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:57:10.133445 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20fa6fc-5222-4fc2-a9c6-b4296a2aeebc admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:57:10.137999 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6d9a3-4bb0-4d5b-ab57-15e737bea4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:57:10.139150 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6d9a3-4bb0-4d5b-ab57-15e737bea4d8 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:57:10.139422 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6d9a3-4bb0-4d5b-ab57-15e737bea4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:57:10.139695 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6d9a3-4bb0-4d5b-ab57-15e737bea4d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:57:10.140442 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d6d9a3-4bb0-4d5b-ab57-15e737bea4d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:57:10.142060 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d6d9a3-4bb0-4d5b-ab57-15e737bea4d8 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:57:10.152737 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1358/2725] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 13:57:10.157610 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d28849f-1a5e-46bb-a653-858bf95a6f25 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:57:10.167665 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d28849f-1a5e-46bb-a653-858bf95a6f25 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:57:10.172531 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43c12c7-f07b-4b4c-aabf-da52c54faf5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:57:10.173220 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c12c7-f07b-4b4c-aabf-da52c54faf5b admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:57:10.173508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c12c7-f07b-4b4c-aabf-da52c54faf5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:57:10.173791 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c12c7-f07b-4b4c-aabf-da52c54faf5b admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:57:10.174516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43c12c7-f07b-4b4c-aabf-da52c54faf5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:57:10.176442 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43c12c7-f07b-4b4c-aabf-da52c54faf5b admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:57:10.186828 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1368/2726] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 13:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 13:57:10.191010 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6d9a3-4bb0-4d5b-ab57-15e737bea4d8 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:57:10.201875 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6d9a3-4bb0-4d5b-ab57-15e737bea4d8 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:57:10.207216 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d9d8b-6cbe-4f67-95d5-f21cd66c4f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:57:10.208036 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d9d8b-6cbe-4f67-95d5-f21cd66c4f34 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:57:10.208315 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d9d8b-6cbe-4f67-95d5-f21cd66c4f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:57:10.208585 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d9d8b-6cbe-4f67-95d5-f21cd66c4f34 admin admin] PATH_INFO: `/v3/projects/4349af63ad274dbfad2615fc50446ded/users/cce90c0e516c4cc0874e43b14182aa61/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:57:10.223895 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d9d8b-6cbe-4f67-95d5-f21cd66c4f34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4349af63ad274dbfad2615fc50446ded, user_id=cce90c0e516c4cc0874e43b14182aa61, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:57:10.226970 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d9d8b-6cbe-4f67-95d5-f21cd66c4f34 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:57:10.240397 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1359/2727] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 13:57:10 2022] PUT /identity/v3/projects/4349af63ad274dbfad2615fc50446ded/users/cce90c0e516c4cc0874e43b14182aa61/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:57:10.245603 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d635e75-08dd-4463-96b4-28ccb843d25d None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:57:10.245902 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d635e75-08dd-4463-96b4-28ccb843d25d None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:57:10.246185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d635e75-08dd-4463-96b4-28ccb843d25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:57:10.265028 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-6d635e75-08dd-4463-96b4-28ccb843d25d None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:57:10.266295 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-6d635e75-08dd-4463-96b4-28ccb843d25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:57:10.274688 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:57:10.331015 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1369/2728] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:57:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:57:10.340675 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d9d8b-6cbe-4f67-95d5-f21cd66c4f34 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:57:10.342894 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-007d9d8b-6cbe-4f67-95d5-f21cd66c4f34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:57:10.400734 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d9d8b-6cbe-4f67-95d5-f21cd66c4f34 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:57:10.404352 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a06bc-c2e1-43f5-8c2b-9deb4930274b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:57:10.404983 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a06bc-c2e1-43f5-8c2b-9deb4930274b service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:57:10.405369 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a06bc-c2e1-43f5-8c2b-9deb4930274b service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:57:10.405650 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a06bc-c2e1-43f5-8c2b-9deb4930274b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:57:10.414650 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a06bc-c2e1-43f5-8c2b-9deb4930274b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:57:10.416087 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a06bc-c2e1-43f5-8c2b-9deb4930274b service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:57:10.426278 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1360/2729] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:57:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:57:11.042757 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3ac1b-b2dd-46c0-a23d-83ef7b972713 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:57:11.043048 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3ac1b-b2dd-46c0-a23d-83ef7b972713 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:57:11.044328 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3ac1b-b2dd-46c0-a23d-83ef7b972713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:57:11.071321 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-edd3ac1b-b2dd-46c0-a23d-83ef7b972713 None None] MFA Rules not processed for user `cce90c0e516c4cc0874e43b14182aa61`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:57:11.072533 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-edd3ac1b-b2dd-46c0-a23d-83ef7b972713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:57:11.146297 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1370/2730] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:57:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:57:11.156959 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a06bc-c2e1-43f5-8c2b-9deb4930274b service neutron] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:57:11.158460 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-179a06bc-c2e1-43f5-8c2b-9deb4930274b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:57:11.217758 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a06bc-c2e1-43f5-8c2b-9deb4930274b service neutron] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:57:11.224329 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de074d5-83e2-4030-a0d5-924be3796a69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:57:11.224329 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de074d5-83e2-4030-a0d5-924be3796a69 service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:57:11.224329 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de074d5-83e2-4030-a0d5-924be3796a69 service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:57:11.224329 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de074d5-83e2-4030-a0d5-924be3796a69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:57:11.232275 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de074d5-83e2-4030-a0d5-924be3796a69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:57:11.233934 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de074d5-83e2-4030-a0d5-924be3796a69 service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:57:11.243122 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1361/2731] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:57:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:57:12.648817 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683f8e4-6829-4c39-acaf-a67c0af38eb0 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:57:12.649450 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683f8e4-6829-4c39-acaf-a67c0af38eb0 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:57:12.649450 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683f8e4-6829-4c39-acaf-a67c0af38eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:57:12.670075 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-e683f8e4-6829-4c39-acaf-a67c0af38eb0 None None] MFA Rules not processed for user `cce90c0e516c4cc0874e43b14182aa61`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:57:12.672029 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-e683f8e4-6829-4c39-acaf-a67c0af38eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:57:12.679734 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1371/2732] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:57:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:57:21.238915 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:DELETE=4,keystone:SELECT=86,keystone:INSERT=3 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:57:22.662018 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=56 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:57:32.863573 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de074d5-83e2-4030-a0d5-924be3796a69 service cinder] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:57:32.864992 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-2de074d5-83e2-4030-a0d5-924be3796a69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:57:32.879198 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de074d5-83e2-4030-a0d5-924be3796a69 service cinder] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:57:32.883827 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2c9b3-b7fb-4ae8-9ade-0fa2a91a8809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:57:32.884703 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2c9b3-b7fb-4ae8-9ade-0fa2a91a8809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:57:32.884980 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2c9b3-b7fb-4ae8-9ade-0fa2a91a8809 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:57:32.885284 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2c9b3-b7fb-4ae8-9ade-0fa2a91a8809 admin admin] PATH_INFO: `/v3/users/0b4c1257a02d471d84a5a04cc027f992` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:57:32.886458 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c2c9b3-b7fb-4ae8-9ade-0fa2a91a8809 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b4c1257a02d471d84a5a04cc027f992)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:57:32.889234 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c2c9b3-b7fb-4ae8-9ade-0fa2a91a8809 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:57:32.937613 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-d0c2c9b3-b7fb-4ae8-9ade-0fa2a91a8809 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b4c1257a02d471d84a5a04cc027f992'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:57:32.945402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-d0c2c9b3-b7fb-4ae8-9ade-0fa2a91a8809 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b4c1257a02d471d84a5a04cc027f992'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:57:32.946030 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-d0c2c9b3-b7fb-4ae8-9ade-0fa2a91a8809 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b4c1257a02d471d84a5a04cc027f992'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:57:32.954156 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-d0c2c9b3-b7fb-4ae8-9ade-0fa2a91a8809 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b4c1257a02d471d84a5a04cc027f992'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:57:32.965851 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-d0c2c9b3-b7fb-4ae8-9ade-0fa2a91a8809 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b4c1257a02d471d84a5a04cc027f992'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:57:32.972281 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1362/2733] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:57:32 2022] DELETE /identity/v3/users/0b4c1257a02d471d84a5a04cc027f992 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:57:32.976706 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebd8b1-5d02-4a7f-8acf-1921b7ee53ba None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:57:32.977243 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebd8b1-5d02-4a7f-8acf-1921b7ee53ba None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:57:32.977352 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebd8b1-5d02-4a7f-8acf-1921b7ee53ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:57:32.995069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-e5ebd8b1-5d02-4a7f-8acf-1921b7ee53ba None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:57:32.996248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-e5ebd8b1-5d02-4a7f-8acf-1921b7ee53ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:57:33.056447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1372/2734] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:57:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:57:33.067071 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2c9b3-b7fb-4ae8-9ade-0fa2a91a8809 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:57:33.068480 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-d0c2c9b3-b7fb-4ae8-9ade-0fa2a91a8809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:57:33.127663 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2c9b3-b7fb-4ae8-9ade-0fa2a91a8809 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:57:33.131369 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3f8e9a-d464-4beb-8bf1-61b5c4d44346 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:57:33.132053 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f8e9a-d464-4beb-8bf1-61b5c4d44346 service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:57:33.132290 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f8e9a-d464-4beb-8bf1-61b5c4d44346 service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:57:33.132567 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f8e9a-d464-4beb-8bf1-61b5c4d44346 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:57:33.142682 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3f8e9a-d464-4beb-8bf1-61b5c4d44346 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:57:33.144163 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3f8e9a-d464-4beb-8bf1-61b5c4d44346 service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:57:33.154062 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1363/2735] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:57:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:57:33.434045 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebd8b1-5d02-4a7f-8acf-1921b7ee53ba None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:57:33.436215 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-e5ebd8b1-5d02-4a7f-8acf-1921b7ee53ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:57:33.451195 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebd8b1-5d02-4a7f-8acf-1921b7ee53ba None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:57:33.455888 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cfaba2-d98a-4d37-bcc4-cae97baaf02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:57:33.456767 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cfaba2-d98a-4d37-bcc4-cae97baaf02b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:57:33.457071 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cfaba2-d98a-4d37-bcc4-cae97baaf02b admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:57:33.457354 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cfaba2-d98a-4d37-bcc4-cae97baaf02b admin admin] PATH_INFO: `/v3/projects/880a0217d7654bbaaf448404cca1b62c` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:57:33.459499 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cfaba2-d98a-4d37-bcc4-cae97baaf02b admin admin] RBAC: Authorizing `identity:delete_project(project_id=880a0217d7654bbaaf448404cca1b62c)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:57:33.462273 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cfaba2-d98a-4d37-bcc4-cae97baaf02b admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:57:33.513916 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-40cfaba2-d98a-4d37-bcc4-cae97baaf02b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '880a0217d7654bbaaf448404cca1b62c', 'request_id': 'req-40cfaba2-d98a-4d37-bcc4-cae97baaf02b'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:57:33.519458 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-40cfaba2-d98a-4d37-bcc4-cae97baaf02b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '880a0217d7654bbaaf448404cca1b62c', 'request_id': 'req-40cfaba2-d98a-4d37-bcc4-cae97baaf02b'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:57:33.526951 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-40cfaba2-d98a-4d37-bcc4-cae97baaf02b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '880a0217d7654bbaaf448404cca1b62c', 'request_id': 'req-40cfaba2-d98a-4d37-bcc4-cae97baaf02b'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:57:33.539099 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-40cfaba2-d98a-4d37-bcc4-cae97baaf02b admin admin] The token cache is being invalidate because project 880a0217d7654bbaaf448404cca1b62c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:57:33.539543 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-40cfaba2-d98a-4d37-bcc4-cae97baaf02b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:57:33.540193 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-40cfaba2-d98a-4d37-bcc4-cae97baaf02b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:57:33.541759 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1373/2736] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:57:33 2022] DELETE /identity/v3/projects/880a0217d7654bbaaf448404cca1b62c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:57:43.149390 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=55,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:57:43.535948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:57:55.528963 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c870a48-73e1-45a4-9cea-5604cbc15303 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:57:55.529892 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c870a48-73e1-45a4-9cea-5604cbc15303 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:57:55.529892 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c870a48-73e1-45a4-9cea-5604cbc15303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:57:55.557468 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-9c870a48-73e1-45a4-9cea-5604cbc15303 None None] MFA Rules not processed for user `c85251638bc64deca16008b696b11ea4`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:57:55.558718 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-9c870a48-73e1-45a4-9cea-5604cbc15303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:57:55.630607 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1364/2737] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:57:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:57:55.967759 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cfaba2-d98a-4d37-bcc4-cae97baaf02b admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:57:55.969342 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-40cfaba2-d98a-4d37-bcc4-cae97baaf02b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:57:56.083000 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cfaba2-d98a-4d37-bcc4-cae97baaf02b admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:57:56.088044 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173172cb-6cdf-43a4-b03d-1684b1593ee3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:57:56.089325 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173172cb-6cdf-43a4-b03d-1684b1593ee3 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:57:56.089666 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173172cb-6cdf-43a4-b03d-1684b1593ee3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:57:56.090015 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173172cb-6cdf-43a4-b03d-1684b1593ee3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:57:56.092407 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-173172cb-6cdf-43a4-b03d-1684b1593ee3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:57:56.104993 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173172cb-6cdf-43a4-b03d-1684b1593ee3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:57:56.106441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173172cb-6cdf-43a4-b03d-1684b1593ee3 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:57:56.118394 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1374/2738] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:57:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:58:05.628507 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=37 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:58:06.113960 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=44 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:58:11.656123 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c870a48-73e1-45a4-9cea-5604cbc15303 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:58:11.657623 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-9c870a48-73e1-45a4-9cea-5604cbc15303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:58:11.745455 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c870a48-73e1-45a4-9cea-5604cbc15303 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:58:11.755504 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb28dd-271c-4a01-bd7c-ddcdeeebcbac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:58:11.756160 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb28dd-271c-4a01-bd7c-ddcdeeebcbac service nova] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:58:11.756446 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb28dd-271c-4a01-bd7c-ddcdeeebcbac service nova] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:58:11.756723 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb28dd-271c-4a01-bd7c-ddcdeeebcbac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:58:11.758777 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-0afb28dd-271c-4a01-bd7c-ddcdeeebcbac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:58:11.770649 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb28dd-271c-4a01-bd7c-ddcdeeebcbac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:58:11.772171 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb28dd-271c-4a01-bd7c-ddcdeeebcbac service nova] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:58:11.780989 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1365/2739] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:58:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:58:21.776933 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=40 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:58:43.181943 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c2d93-75a1-45d6-93d2-ef6a7139a0b3 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:58:43.182491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c2d93-75a1-45d6-93d2-ef6a7139a0b3 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:58:43.182491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c2d93-75a1-45d6-93d2-ef6a7139a0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:58:43.199629 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-453c2d93-75a1-45d6-93d2-ef6a7139a0b3 None None] MFA Rules not processed for user `cce90c0e516c4cc0874e43b14182aa61`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 13:58:43.200805 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-453c2d93-75a1-45d6-93d2-ef6a7139a0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:58:43.253660 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1375/2740] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 13:58:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 13:58:43.749276 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb28dd-271c-4a01-bd7c-ddcdeeebcbac service nova] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:58:43.750752 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-0afb28dd-271c-4a01-bd7c-ddcdeeebcbac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:58:43.762775 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb28dd-271c-4a01-bd7c-ddcdeeebcbac service nova] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:58:43.766311 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7426c2f-be4c-4a9f-88fe-e7d61f92d746 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:58:43.766946 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7426c2f-be4c-4a9f-88fe-e7d61f92d746 service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:58:43.767233 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7426c2f-be4c-4a9f-88fe-e7d61f92d746 service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:58:43.767513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7426c2f-be4c-4a9f-88fe-e7d61f92d746 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:58:43.769225 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-c7426c2f-be4c-4a9f-88fe-e7d61f92d746 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:58:43.779180 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7426c2f-be4c-4a9f-88fe-e7d61f92d746 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:58:43.780602 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7426c2f-be4c-4a9f-88fe-e7d61f92d746 service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:58:43.788616 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1366/2741] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:58:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:58:53.247207 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=25 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:58:53.784960 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=6 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:59:32.340245 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453c2d93-75a1-45d6-93d2-ef6a7139a0b3 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:59:32.341807 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-453c2d93-75a1-45d6-93d2-ef6a7139a0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:59:32.443892 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453c2d93-75a1-45d6-93d2-ef6a7139a0b3 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:59:32.444187 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040abf01-3398-4cfb-b8ae-83061960107b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '890d2ae150d647a18dae25159cf97484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:59:32.445095 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040abf01-3398-4cfb-b8ae-83061960107b service placement] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:59:32.445351 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040abf01-3398-4cfb-b8ae-83061960107b service placement] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:59:32.445783 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040abf01-3398-4cfb-b8ae-83061960107b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:59:32.447733 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-040abf01-3398-4cfb-b8ae-83061960107b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:59:32.461137 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040abf01-3398-4cfb-b8ae-83061960107b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:59:32.463053 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040abf01-3398-4cfb-b8ae-83061960107b service placement] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:59:32.474309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1376/2742] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:59:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:59:35.723602 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7426c2f-be4c-4a9f-88fe-e7d61f92d746 service neutron] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:59:35.732275 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7426c2f-be4c-4a9f-88fe-e7d61f92d746 service neutron] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:59:35.735880 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a831d-2d37-4a2b-9921-a288998d1f02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:59:35.736518 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a831d-2d37-4a2b-9921-a288998d1f02 service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:59:35.736801 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a831d-2d37-4a2b-9921-a288998d1f02 service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:59:35.737111 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a831d-2d37-4a2b-9921-a288998d1f02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:59:35.738882 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-319a831d-2d37-4a2b-9921-a288998d1f02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:59:35.823033 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319a831d-2d37-4a2b-9921-a288998d1f02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:59:35.824388 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319a831d-2d37-4a2b-9921-a288998d1f02 service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:59:35.843880 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1367/2743] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:59:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:59:38.838647 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040abf01-3398-4cfb-b8ae-83061960107b service placement] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:59:38.849708 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040abf01-3398-4cfb-b8ae-83061960107b service placement] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:59:38.855664 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e3189-6a21-42fd-839f-874d2280c484 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '890d2ae150d647a18dae25159cf97484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:59:38.856484 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e3189-6a21-42fd-839f-874d2280c484 service placement] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:59:38.856833 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e3189-6a21-42fd-839f-874d2280c484 service placement] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:59:38.857208 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e3189-6a21-42fd-839f-874d2280c484 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:59:38.859026 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-e31e3189-6a21-42fd-839f-874d2280c484 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:59:38.871657 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31e3189-6a21-42fd-839f-874d2280c484 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:59:38.873347 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31e3189-6a21-42fd-839f-874d2280c484 service placement] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:59:38.883391 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1377/2744] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 13:59:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 13:59:41.966635 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a831d-2d37-4a2b-9921-a288998d1f02 service neutron] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:59:41.968111 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-319a831d-2d37-4a2b-9921-a288998d1f02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:59:41.984259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a831d-2d37-4a2b-9921-a288998d1f02 service neutron] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:59:41.992879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e013ee1-2b44-464b-aea6-fee7f6c8561f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:59:41.994152 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e013ee1-2b44-464b-aea6-fee7f6c8561f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:59:41.994518 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e013ee1-2b44-464b-aea6-fee7f6c8561f admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:59:41.995546 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e013ee1-2b44-464b-aea6-fee7f6c8561f admin admin] PATH_INFO: `/v3/users/32a8191b1c184c75b428579ace2da61c` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:59:41.997337 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e013ee1-2b44-464b-aea6-fee7f6c8561f admin admin] RBAC: Authorizing `identity:delete_user(user_id=32a8191b1c184c75b428579ace2da61c)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:59:41.999485 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e013ee1-2b44-464b-aea6-fee7f6c8561f admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:59:42.050087 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-0e013ee1-2b44-464b-aea6-fee7f6c8561f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32a8191b1c184c75b428579ace2da61c'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:59:42.058794 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-0e013ee1-2b44-464b-aea6-fee7f6c8561f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32a8191b1c184c75b428579ace2da61c'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:59:42.059344 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-0e013ee1-2b44-464b-aea6-fee7f6c8561f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32a8191b1c184c75b428579ace2da61c'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:59:42.066306 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-0e013ee1-2b44-464b-aea6-fee7f6c8561f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32a8191b1c184c75b428579ace2da61c'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:59:42.078490 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-0e013ee1-2b44-464b-aea6-fee7f6c8561f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32a8191b1c184c75b428579ace2da61c'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:59:42.084286 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1368/2745] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:59:41 2022] DELETE /identity/v3/users/32a8191b1c184c75b428579ace2da61c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:59:42.263844 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e3189-6a21-42fd-839f-874d2280c484 service placement] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:59:42.276361 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e3189-6a21-42fd-839f-874d2280c484 service placement] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:59:42.338481 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19284f53-f381-4e81-a350-0230dd821783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:59:42.339293 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19284f53-f381-4e81-a350-0230dd821783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:59:42.339556 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19284f53-f381-4e81-a350-0230dd821783 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:59:42.339833 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19284f53-f381-4e81-a350-0230dd821783 admin admin] PATH_INFO: `/v3/projects/7bf864bb1764452e8e3a6f0d5929da9f` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:59:42.341049 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19284f53-f381-4e81-a350-0230dd821783 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bf864bb1764452e8e3a6f0d5929da9f)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:59:42.342954 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19284f53-f381-4e81-a350-0230dd821783 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:59:42.392856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-19284f53-f381-4e81-a350-0230dd821783 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bf864bb1764452e8e3a6f0d5929da9f', 'request_id': 'req-19284f53-f381-4e81-a350-0230dd821783'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:59:42.398318 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-19284f53-f381-4e81-a350-0230dd821783 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bf864bb1764452e8e3a6f0d5929da9f', 'request_id': 'req-19284f53-f381-4e81-a350-0230dd821783'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:59:42.399905 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:59:42.405613 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-19284f53-f381-4e81-a350-0230dd821783 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bf864bb1764452e8e3a6f0d5929da9f', 'request_id': 'req-19284f53-f381-4e81-a350-0230dd821783'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:59:42.417248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-19284f53-f381-4e81-a350-0230dd821783 admin admin] The token cache is being invalidate because project 7bf864bb1764452e8e3a6f0d5929da9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:59:42.417532 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-19284f53-f381-4e81-a350-0230dd821783 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:59:42.418161 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-19284f53-f381-4e81-a350-0230dd821783 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:59:42.419755 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1378/2746] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:59:42 2022] DELETE /identity/v3/projects/7bf864bb1764452e8e3a6f0d5929da9f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:59:42.423686 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e013ee1-2b44-464b-aea6-fee7f6c8561f admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:59:42.425088 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-0e013ee1-2b44-464b-aea6-fee7f6c8561f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:59:42.438885 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e013ee1-2b44-464b-aea6-fee7f6c8561f admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:59:42.471533 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:59:42.497571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d31de0-e1b9-4a12-9330-4af925c1e721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:59:42.498476 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d31de0-e1b9-4a12-9330-4af925c1e721 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:59:42.498759 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d31de0-e1b9-4a12-9330-4af925c1e721 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:59:42.499034 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d31de0-e1b9-4a12-9330-4af925c1e721 admin admin] PATH_INFO: `/v3/users/83c2308390c64052984b7150f85838a7` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:59:42.500216 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d31de0-e1b9-4a12-9330-4af925c1e721 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83c2308390c64052984b7150f85838a7)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:59:42.502163 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d31de0-e1b9-4a12-9330-4af925c1e721 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:59:42.549694 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-f0d31de0-e1b9-4a12-9330-4af925c1e721 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83c2308390c64052984b7150f85838a7'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:59:42.560468 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-f0d31de0-e1b9-4a12-9330-4af925c1e721 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83c2308390c64052984b7150f85838a7'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:59:42.561112 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-f0d31de0-e1b9-4a12-9330-4af925c1e721 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83c2308390c64052984b7150f85838a7'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:59:42.568225 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-f0d31de0-e1b9-4a12-9330-4af925c1e721 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83c2308390c64052984b7150f85838a7'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:59:42.586164 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-f0d31de0-e1b9-4a12-9330-4af925c1e721 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83c2308390c64052984b7150f85838a7'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:59:42.593682 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1369/2747] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:59:42 2022] DELETE /identity/v3/users/83c2308390c64052984b7150f85838a7 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:59:42.766776 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19284f53-f381-4e81-a350-0230dd821783 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:59:42.768315 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-19284f53-f381-4e81-a350-0230dd821783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:59:42.798605 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19284f53-f381-4e81-a350-0230dd821783 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:59:42.858984 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff4773-9972-4570-9c76-1928d68c6136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:59:42.859843 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff4773-9972-4570-9c76-1928d68c6136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:59:42.860237 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff4773-9972-4570-9c76-1928d68c6136 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:59:42.860534 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff4773-9972-4570-9c76-1928d68c6136 admin admin] PATH_INFO: `/v3/projects/ca371b04b33244adae871d37770223d9` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:59:42.861737 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efff4773-9972-4570-9c76-1928d68c6136 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca371b04b33244adae871d37770223d9)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:59:42.863590 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efff4773-9972-4570-9c76-1928d68c6136 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:59:42.922152 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-efff4773-9972-4570-9c76-1928d68c6136 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca371b04b33244adae871d37770223d9', 'request_id': 'req-efff4773-9972-4570-9c76-1928d68c6136'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:59:42.929838 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-efff4773-9972-4570-9c76-1928d68c6136 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca371b04b33244adae871d37770223d9', 'request_id': 'req-efff4773-9972-4570-9c76-1928d68c6136'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:59:42.937285 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-efff4773-9972-4570-9c76-1928d68c6136 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca371b04b33244adae871d37770223d9', 'request_id': 'req-efff4773-9972-4570-9c76-1928d68c6136'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:59:42.949194 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-efff4773-9972-4570-9c76-1928d68c6136 admin admin] The token cache is being invalidate because project ca371b04b33244adae871d37770223d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:59:42.949481 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-efff4773-9972-4570-9c76-1928d68c6136 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:59:42.950111 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-efff4773-9972-4570-9c76-1928d68c6136 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:59:42.951640 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1379/2748] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:59:42 2022] DELETE /identity/v3/projects/ca371b04b33244adae871d37770223d9 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:59:51.409581 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d31de0-e1b9-4a12-9330-4af925c1e721 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:59:51.411054 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-f0d31de0-e1b9-4a12-9330-4af925c1e721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:59:51.425657 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d31de0-e1b9-4a12-9330-4af925c1e721 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:59:51.482184 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e733182-d253-4b1d-868c-6391f483d7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:59:51.483050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e733182-d253-4b1d-868c-6391f483d7a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:59:51.483333 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e733182-d253-4b1d-868c-6391f483d7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:59:51.483608 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e733182-d253-4b1d-868c-6391f483d7a9 admin admin] PATH_INFO: `/v3/users/cce90c0e516c4cc0874e43b14182aa61` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:59:51.484784 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e733182-d253-4b1d-868c-6391f483d7a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cce90c0e516c4cc0874e43b14182aa61)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:59:51.486744 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e733182-d253-4b1d-868c-6391f483d7a9 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:59:51.531210 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-3e733182-d253-4b1d-868c-6391f483d7a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cce90c0e516c4cc0874e43b14182aa61'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:59:51.539464 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-3e733182-d253-4b1d-868c-6391f483d7a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cce90c0e516c4cc0874e43b14182aa61'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:59:51.540093 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-3e733182-d253-4b1d-868c-6391f483d7a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cce90c0e516c4cc0874e43b14182aa61'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:59:51.544740 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:59:51.547618 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-3e733182-d253-4b1d-868c-6391f483d7a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cce90c0e516c4cc0874e43b14182aa61'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:59:51.563182 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-3e733182-d253-4b1d-868c-6391f483d7a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cce90c0e516c4cc0874e43b14182aa61'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:59:51.569254 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1370/2749] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 13:59:51 2022] DELETE /identity/v3/users/cce90c0e516c4cc0874e43b14182aa61 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 13:59:51.739201 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff4773-9972-4570-9c76-1928d68c6136 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 13:59:51.740659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-efff4773-9972-4570-9c76-1928d68c6136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 13:59:51.756310 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff4773-9972-4570-9c76-1928d68c6136 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 13:59:51.814702 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d3c01-e3ed-4243-9eea-93dcfbea2758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 13:59:51.815510 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d3c01-e3ed-4243-9eea-93dcfbea2758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 13:59:51.815792 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d3c01-e3ed-4243-9eea-93dcfbea2758 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 13:59:51.816069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d3c01-e3ed-4243-9eea-93dcfbea2758 admin admin] PATH_INFO: `/v3/projects/4349af63ad274dbfad2615fc50446ded` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 13:59:51.817259 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d3c01-e3ed-4243-9eea-93dcfbea2758 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4349af63ad274dbfad2615fc50446ded)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 13:59:51.819201 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d3c01-e3ed-4243-9eea-93dcfbea2758 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 13:59:51.840601 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 13:59:51.868787 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b61d3c01-e3ed-4243-9eea-93dcfbea2758 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4349af63ad274dbfad2615fc50446ded', 'request_id': 'req-b61d3c01-e3ed-4243-9eea-93dcfbea2758'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:59:51.874754 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b61d3c01-e3ed-4243-9eea-93dcfbea2758 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4349af63ad274dbfad2615fc50446ded', 'request_id': 'req-b61d3c01-e3ed-4243-9eea-93dcfbea2758'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:59:51.882199 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b61d3c01-e3ed-4243-9eea-93dcfbea2758 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4349af63ad274dbfad2615fc50446ded', 'request_id': 'req-b61d3c01-e3ed-4243-9eea-93dcfbea2758'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:59:51.893709 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b61d3c01-e3ed-4243-9eea-93dcfbea2758 admin admin] The token cache is being invalidate because project 4349af63ad274dbfad2615fc50446ded was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 13:59:51.893999 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b61d3c01-e3ed-4243-9eea-93dcfbea2758 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:59:51.894641 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b61d3c01-e3ed-4243-9eea-93dcfbea2758 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 13:59:51.896177 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1380/2750] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 13:59:51 2022] DELETE /identity/v3/projects/4349af63ad274dbfad2615fc50446ded => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:00:01.566019 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=9,keystone:DELETE=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:00:01.890661 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:00:20.473038 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8bf72-4908-43f3-9442-d81b369f6a8a None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:00:20.473513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8bf72-4908-43f3-9442-d81b369f6a8a None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:00:20.473618 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8bf72-4908-43f3-9442-d81b369f6a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:00:20.501079 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-6aa8bf72-4908-43f3-9442-d81b369f6a8a None None] MFA Rules not processed for user `c4535134181e489fb532f617264d5855`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:00:20.502292 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-6aa8bf72-4908-43f3-9442-d81b369f6a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:00:20.559430 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1371/2751] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Jul 6 14:00:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 14:00:20.604022 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d3c01-e3ed-4243-9eea-93dcfbea2758 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:00:20.605462 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-b61d3c01-e3ed-4243-9eea-93dcfbea2758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:00:20.676834 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d3c01-e3ed-4243-9eea-93dcfbea2758 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:00:20.682105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f443d-74b0-4096-91dd-ccd0ad6378ae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c1bea05c37646b0a05fa8dda0367ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:00:20.682703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f443d-74b0-4096-91dd-ccd0ad6378ae service swift] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:00:20.682982 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f443d-74b0-4096-91dd-ccd0ad6378ae service swift] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:00:20.683255 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f443d-74b0-4096-91dd-ccd0ad6378ae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:00:20.691210 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219f443d-74b0-4096-91dd-ccd0ad6378ae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:00:20.692760 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219f443d-74b0-4096-91dd-ccd0ad6378ae service swift] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:00:20.700542 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1381/2752] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jul 6 14:00:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 14:00:30.552566 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=29 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:00:30.697366 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=32 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:00:34.209637 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8bf72-4908-43f3-9442-d81b369f6a8a None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:00:34.212538 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-6aa8bf72-4908-43f3-9442-d81b369f6a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:00:34.280742 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8bf72-4908-43f3-9442-d81b369f6a8a None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:00:34.284426 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b98663e-6d9e-4977-9057-5b7bbce9da59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:00:34.285374 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98663e-6d9e-4977-9057-5b7bbce9da59 service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:00:34.285659 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98663e-6d9e-4977-9057-5b7bbce9da59 service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:00:34.285932 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98663e-6d9e-4977-9057-5b7bbce9da59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:00:34.288028 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-5b98663e-6d9e-4977-9057-5b7bbce9da59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:00:34.298730 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b98663e-6d9e-4977-9057-5b7bbce9da59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:00:34.300157 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b98663e-6d9e-4977-9057-5b7bbce9da59 service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:00:34.319437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1372/2753] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 14:00:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 14:00:34.395839 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f443d-74b0-4096-91dd-ccd0ad6378ae service swift] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:00:34.405277 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f443d-74b0-4096-91dd-ccd0ad6378ae service swift] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:00:34.408734 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f9cbc2-2e0d-45a8-b3f4-122a7a1b035e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:00:34.409396 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9cbc2-2e0d-45a8-b3f4-122a7a1b035e service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:00:34.409680 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9cbc2-2e0d-45a8-b3f4-122a7a1b035e service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:00:34.409956 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9cbc2-2e0d-45a8-b3f4-122a7a1b035e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:00:34.411694 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-14f9cbc2-2e0d-45a8-b3f4-122a7a1b035e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:00:34.492921 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f9cbc2-2e0d-45a8-b3f4-122a7a1b035e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:00:34.494374 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f9cbc2-2e0d-45a8-b3f4-122a7a1b035e service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:00:34.517031 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1382/2754] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 14:00:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 14:00:36.187765 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df885c51-9ef1-480c-b300-86bcb39e03a6 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:00:36.188259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df885c51-9ef1-480c-b300-86bcb39e03a6 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:00:36.188380 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df885c51-9ef1-480c-b300-86bcb39e03a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:00:36.205481 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-df885c51-9ef1-480c-b300-86bcb39e03a6 None None] MFA Rules not processed for user `c4535134181e489fb532f617264d5855`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:00:36.206657 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-df885c51-9ef1-480c-b300-86bcb39e03a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:00:36.213714 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1373/2755] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Jul 6 14:00:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 14:00:36.225245 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f9cbc2-2e0d-45a8-b3f4-122a7a1b035e service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:00:36.236961 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f9cbc2-2e0d-45a8-b3f4-122a7a1b035e service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:00:36.242131 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb36701-4619-47ae-83ba-6999ab2c52af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c1bea05c37646b0a05fa8dda0367ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:00:36.242783 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb36701-4619-47ae-83ba-6999ab2c52af service swift] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:00:36.243067 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb36701-4619-47ae-83ba-6999ab2c52af service swift] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:00:36.243356 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb36701-4619-47ae-83ba-6999ab2c52af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:00:36.252346 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb36701-4619-47ae-83ba-6999ab2c52af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:00:36.253964 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb36701-4619-47ae-83ba-6999ab2c52af service swift] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:00:36.261846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1383/2756] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jul 6 14:00:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 14:00:46.200359 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=29 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:00:46.258640 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=49 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:00:51.964547 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df885c51-9ef1-480c-b300-86bcb39e03a6 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:00:51.974817 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df885c51-9ef1-480c-b300-86bcb39e03a6 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:00:51.978413 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a0fa4-ee2b-4109-a05b-912c4d738a42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:00:51.979055 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a0fa4-ee2b-4109-a05b-912c4d738a42 service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:00:51.979341 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a0fa4-ee2b-4109-a05b-912c4d738a42 service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:00:51.979621 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a0fa4-ee2b-4109-a05b-912c4d738a42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:00:51.981466 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-f27a0fa4-ee2b-4109-a05b-912c4d738a42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:00:51.994995 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27a0fa4-ee2b-4109-a05b-912c4d738a42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:00:51.996441 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27a0fa4-ee2b-4109-a05b-912c4d738a42 service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:00:52.007855 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1374/2757] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 14:00:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 14:00:56.702288 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb36701-4619-47ae-83ba-6999ab2c52af service swift] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:00:56.703758 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-1bb36701-4619-47ae-83ba-6999ab2c52af service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:00:56.715534 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb36701-4619-47ae-83ba-6999ab2c52af service swift] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:00:56.720220 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e70eb-1fa4-4c87-a50b-a9a5891c7449 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:00:56.720856 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e70eb-1fa4-4c87-a50b-a9a5891c7449 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:00:56.721175 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e70eb-1fa4-4c87-a50b-a9a5891c7449 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:00:56.721461 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e70eb-1fa4-4c87-a50b-a9a5891c7449 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:00:56.723168 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-972e70eb-1fa4-4c87-a50b-a9a5891c7449 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:00:56.736546 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e70eb-1fa4-4c87-a50b-a9a5891c7449 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:00:56.738012 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e70eb-1fa4-4c87-a50b-a9a5891c7449 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:00:56.750040 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1384/2758] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 14:00:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 14:00:57.286046 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01375f55-5408-494c-a12c-b9bbb3094eb5 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:00:57.286496 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01375f55-5408-494c-a12c-b9bbb3094eb5 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:00:57.286597 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01375f55-5408-494c-a12c-b9bbb3094eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:00:57.304947 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-01375f55-5408-494c-a12c-b9bbb3094eb5 None None] MFA Rules not processed for user `c4535134181e489fb532f617264d5855`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:00:57.306224 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-01375f55-5408-494c-a12c-b9bbb3094eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:00:57.314013 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1375/2759] 10.222.0.20 () {58 vars in 967 bytes} [Wed Jul 6 14:00:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 14:00:57.326811 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e70eb-1fa4-4c87-a50b-a9a5891c7449 service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:00:57.338270 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e70eb-1fa4-4c87-a50b-a9a5891c7449 service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:00:57.343244 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74fc54-82de-42cd-8894-8eed6c882d55 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c1bea05c37646b0a05fa8dda0367ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:00:57.343882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74fc54-82de-42cd-8894-8eed6c882d55 service swift] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:00:57.344164 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74fc54-82de-42cd-8894-8eed6c882d55 service swift] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:00:57.344435 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74fc54-82de-42cd-8894-8eed6c882d55 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:00:57.352624 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74fc54-82de-42cd-8894-8eed6c882d55 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:00:57.354237 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74fc54-82de-42cd-8894-8eed6c882d55 service swift] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:00:57.363267 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1385/2760] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jul 6 14:00:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 14:01:07.299545 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=9 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:01:07.360011 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=12 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:01:34.149464 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01375f55-5408-494c-a12c-b9bbb3094eb5 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:01:34.158938 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01375f55-5408-494c-a12c-b9bbb3094eb5 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:01:34.162439 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d92a1-000b-4663-8c6e-464b2bbda7ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:01:34.163077 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d92a1-000b-4663-8c6e-464b2bbda7ed service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:01:34.163357 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d92a1-000b-4663-8c6e-464b2bbda7ed service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:01:34.163632 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d92a1-000b-4663-8c6e-464b2bbda7ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:01:34.165447 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-273d92a1-000b-4663-8c6e-464b2bbda7ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:01:34.175404 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d92a1-000b-4663-8c6e-464b2bbda7ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:01:34.176805 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d92a1-000b-4663-8c6e-464b2bbda7ed service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:01:34.184995 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1376/2761] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 14:01:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 14:01:44.181388 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=6 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:01:55.846749 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74fc54-82de-42cd-8894-8eed6c882d55 service swift] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:01:55.849331 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-5e74fc54-82de-42cd-8894-8eed6c882d55 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:01:55.875438 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74fc54-82de-42cd-8894-8eed6c882d55 service swift] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:01:55.881353 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712dce85-13ff-45d3-bc0e-d76287063f0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6647eb1e8a1544ef9d3a9806f2d1fc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:01:55.882048 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712dce85-13ff-45d3-bc0e-d76287063f0a service nova] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:01:55.882339 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712dce85-13ff-45d3-bc0e-d76287063f0a service nova] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:01:55.882613 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712dce85-13ff-45d3-bc0e-d76287063f0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:01:55.884357 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-712dce85-13ff-45d3-bc0e-d76287063f0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:01:55.897855 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712dce85-13ff-45d3-bc0e-d76287063f0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:01:55.899508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712dce85-13ff-45d3-bc0e-d76287063f0a service nova] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:01:55.911739 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1386/2762] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 14:01:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 14:01:59.946601 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d92a1-000b-4663-8c6e-464b2bbda7ed service neutron] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:01:59.955165 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d92a1-000b-4663-8c6e-464b2bbda7ed service neutron] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:01:59.958682 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc21f5-4344-49e9-b564-54acd451b277 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:01:59.959310 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc21f5-4344-49e9-b564-54acd451b277 service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:01:59.959591 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc21f5-4344-49e9-b564-54acd451b277 service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:01:59.959865 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc21f5-4344-49e9-b564-54acd451b277 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:01:59.961611 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-86cc21f5-4344-49e9-b564-54acd451b277 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:00.027449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc21f5-4344-49e9-b564-54acd451b277 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:00.030010 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc21f5-4344-49e9-b564-54acd451b277 service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:00.039709 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1377/2763] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 14:01:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 14:02:03.274580 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712dce85-13ff-45d3-bc0e-d76287063f0a service nova] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:03.275952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-712dce85-13ff-45d3-bc0e-d76287063f0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:03.290502 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712dce85-13ff-45d3-bc0e-d76287063f0a service nova] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:03.297869 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf23928-c2ef-4d8c-b1e1-04b1ce21c196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:03.299545 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf23928-c2ef-4d8c-b1e1-04b1ce21c196 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:03.299847 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf23928-c2ef-4d8c-b1e1-04b1ce21c196 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:03.300119 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf23928-c2ef-4d8c-b1e1-04b1ce21c196 admin admin] PATH_INFO: `/v3/users/c85251638bc64deca16008b696b11ea4` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:03.301713 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf23928-c2ef-4d8c-b1e1-04b1ce21c196 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c85251638bc64deca16008b696b11ea4)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:03.303915 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf23928-c2ef-4d8c-b1e1-04b1ce21c196 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:03.355724 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-7bf23928-c2ef-4d8c-b1e1-04b1ce21c196 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c85251638bc64deca16008b696b11ea4'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:03.364812 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-7bf23928-c2ef-4d8c-b1e1-04b1ce21c196 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c85251638bc64deca16008b696b11ea4'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:03.373301 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-7bf23928-c2ef-4d8c-b1e1-04b1ce21c196 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c85251638bc64deca16008b696b11ea4'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:03.374043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-7bf23928-c2ef-4d8c-b1e1-04b1ce21c196 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c85251638bc64deca16008b696b11ea4'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:03.386941 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-7bf23928-c2ef-4d8c-b1e1-04b1ce21c196 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c85251638bc64deca16008b696b11ea4'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:03.394380 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1387/2764] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 14:02:03 2022] DELETE /identity/v3/users/c85251638bc64deca16008b696b11ea4 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:03.594090 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc21f5-4344-49e9-b564-54acd451b277 service neutron] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:03.604513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc21f5-4344-49e9-b564-54acd451b277 service neutron] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:03.663874 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d1ea1-4c14-434a-892e-d69860d4681c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:03.664739 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d1ea1-4c14-434a-892e-d69860d4681c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:03.665049 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d1ea1-4c14-434a-892e-d69860d4681c admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:03.665334 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d1ea1-4c14-434a-892e-d69860d4681c admin admin] PATH_INFO: `/v3/projects/9e3edb8d53184b98b7055ffc145f186d` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:03.666510 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8d1ea1-4c14-434a-892e-d69860d4681c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e3edb8d53184b98b7055ffc145f186d)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:03.668437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8d1ea1-4c14-434a-892e-d69860d4681c admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:03.714242 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-df8d1ea1-4c14-434a-892e-d69860d4681c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e3edb8d53184b98b7055ffc145f186d', 'request_id': 'req-df8d1ea1-4c14-434a-892e-d69860d4681c'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:03.720658 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-df8d1ea1-4c14-434a-892e-d69860d4681c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e3edb8d53184b98b7055ffc145f186d', 'request_id': 'req-df8d1ea1-4c14-434a-892e-d69860d4681c'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:03.728131 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-df8d1ea1-4c14-434a-892e-d69860d4681c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e3edb8d53184b98b7055ffc145f186d', 'request_id': 'req-df8d1ea1-4c14-434a-892e-d69860d4681c'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:03.738936 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-df8d1ea1-4c14-434a-892e-d69860d4681c admin admin] The token cache is being invalidate because project 9e3edb8d53184b98b7055ffc145f186d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 14:02:03.739221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-df8d1ea1-4c14-434a-892e-d69860d4681c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:03.739831 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-df8d1ea1-4c14-434a-892e-d69860d4681c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:03.741393 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1378/2765] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 14:02:03 2022] DELETE /identity/v3/projects/9e3edb8d53184b98b7055ffc145f186d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:03.746278 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b9dbf-f1b7-45fc-a254-55a24b2c70d6 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:03.746565 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b9dbf-f1b7-45fc-a254-55a24b2c70d6 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:03.746845 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b9dbf-f1b7-45fc-a254-55a24b2c70d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:03.802269 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-b18b9dbf-f1b7-45fc-a254-55a24b2c70d6 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:02:03.803652 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-b18b9dbf-f1b7-45fc-a254-55a24b2c70d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:03.866929 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1388/2766] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 14:02:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 14:02:03.874704 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2647422-503c-4146-bec8-bcc69761079b None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:03.874998 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2647422-503c-4146-bec8-bcc69761079b None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:03.875276 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2647422-503c-4146-bec8-bcc69761079b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:03.893813 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-d2647422-503c-4146-bec8-bcc69761079b None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:02:03.895016 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-d2647422-503c-4146-bec8-bcc69761079b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:03.902893 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1379/2767] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 14:02:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 14:02:03.907595 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b9dbf-f1b7-45fc-a254-55a24b2c70d6 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:03.917209 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b9dbf-f1b7-45fc-a254-55a24b2c70d6 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:03.922638 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dded1e-7b96-4f9c-9d86-3bc7773c71a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:03.923291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dded1e-7b96-4f9c-9d86-3bc7773c71a7 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:03.923576 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dded1e-7b96-4f9c-9d86-3bc7773c71a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:03.923855 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dded1e-7b96-4f9c-9d86-3bc7773c71a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:03.924580 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dded1e-7b96-4f9c-9d86-3bc7773c71a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 14:02:03.924890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dded1e-7b96-4f9c-9d86-3bc7773c71a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:03.926508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dded1e-7b96-4f9c-9d86-3bc7773c71a7 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:03.938350 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1389/2768] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 14:02:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 14:02:03.942511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2647422-503c-4146-bec8-bcc69761079b None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:03.953105 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2647422-503c-4146-bec8-bcc69761079b None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:03.959206 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a76aa-e694-4062-878e-17675060fa48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:03.959860 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a76aa-e694-4062-878e-17675060fa48 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:03.960149 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a76aa-e694-4062-878e-17675060fa48 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:03.960430 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a76aa-e694-4062-878e-17675060fa48 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:03.961162 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a76aa-e694-4062-878e-17675060fa48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:03.963085 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a76aa-e694-4062-878e-17675060fa48 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:03.978032 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:03.992291 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1380/2769] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 14:02:03 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 14:02:03.996350 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dded1e-7b96-4f9c-9d86-3bc7773c71a7 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:04.006845 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dded1e-7b96-4f9c-9d86-3bc7773c71a7 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:04.019004 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:04.070190 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68acab-a9bd-4dc7-98ca-2a83d9fd0b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:04.070779 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68acab-a9bd-4dc7-98ca-2a83d9fd0b99 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:04.071061 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68acab-a9bd-4dc7-98ca-2a83d9fd0b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:04.071337 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68acab-a9bd-4dc7-98ca-2a83d9fd0b99 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:04.072023 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a68acab-a9bd-4dc7-98ca-2a83d9fd0b99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:04.073971 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a68acab-a9bd-4dc7-98ca-2a83d9fd0b99 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:04.094505 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1390/2770] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 14:02:03 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 14:02:04.101491 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a76aa-e694-4062-878e-17675060fa48 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:04.113496 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a76aa-e694-4062-878e-17675060fa48 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:04.118111 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac33244-8c6e-476a-9bee-0e698f18ef12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:04.118781 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac33244-8c6e-476a-9bee-0e698f18ef12 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:04.119061 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac33244-8c6e-476a-9bee-0e698f18ef12 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:04.119339 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac33244-8c6e-476a-9bee-0e698f18ef12 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:04.120064 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac33244-8c6e-476a-9bee-0e698f18ef12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:04.121694 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac33244-8c6e-476a-9bee-0e698f18ef12 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:04.132390 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1381/2771] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 14:02:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 14:02:04.136488 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68acab-a9bd-4dc7-98ca-2a83d9fd0b99 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:04.147093 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68acab-a9bd-4dc7-98ca-2a83d9fd0b99 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:04.152940 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92250e58-2031-4efc-9392-f00e66ffde19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:04.154300 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92250e58-2031-4efc-9392-f00e66ffde19 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:04.154674 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92250e58-2031-4efc-9392-f00e66ffde19 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:04.154955 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92250e58-2031-4efc-9392-f00e66ffde19 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:04.155678 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92250e58-2031-4efc-9392-f00e66ffde19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:04.157284 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92250e58-2031-4efc-9392-f00e66ffde19 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:04.167359 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1391/2772] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 14:02:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 14:02:04.171409 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac33244-8c6e-476a-9bee-0e698f18ef12 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:04.182090 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac33244-8c6e-476a-9bee-0e698f18ef12 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:04.187612 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f4d1c1-4873-443a-b003-1ff8d0ef1f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:04.188765 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4d1c1-4873-443a-b003-1ff8d0ef1f98 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:04.189072 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4d1c1-4873-443a-b003-1ff8d0ef1f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:04.189346 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4d1c1-4873-443a-b003-1ff8d0ef1f98 admin admin] PATH_INFO: `/v3/projects/39cd367c83b54b17b4fff804f013f953/users/a47c0ebfdb7f4dc2a4da75f9016553a5/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:04.205314 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f4d1c1-4873-443a-b003-1ff8d0ef1f98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39cd367c83b54b17b4fff804f013f953, user_id=a47c0ebfdb7f4dc2a4da75f9016553a5, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:04.207939 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f4d1c1-4873-443a-b003-1ff8d0ef1f98 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:04.221400 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1382/2773] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 14:02:04 2022] PUT /identity/v3/projects/39cd367c83b54b17b4fff804f013f953/users/a47c0ebfdb7f4dc2a4da75f9016553a5/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:04.232420 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32909d61-92cf-4e56-a750-38059d681aac None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:04.232707 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32909d61-92cf-4e56-a750-38059d681aac None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:04.232984 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32909d61-92cf-4e56-a750-38059d681aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:04.258942 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-32909d61-92cf-4e56-a750-38059d681aac None None] MFA Rules not processed for user `a47c0ebfdb7f4dc2a4da75f9016553a5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:02:04.260124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-32909d61-92cf-4e56-a750-38059d681aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:04.323625 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1392/2774] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 14:02:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 14:02:04.328632 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f4d1c1-4873-443a-b003-1ff8d0ef1f98 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:04.390836 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f4d1c1-4873-443a-b003-1ff8d0ef1f98 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:04.395505 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a37974-18fd-4a3a-b56b-77ff8e199580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:04.396158 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a37974-18fd-4a3a-b56b-77ff8e199580 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:04.396445 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a37974-18fd-4a3a-b56b-77ff8e199580 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:04.396725 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a37974-18fd-4a3a-b56b-77ff8e199580 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:04.397470 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a37974-18fd-4a3a-b56b-77ff8e199580 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:04.399368 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a37974-18fd-4a3a-b56b-77ff8e199580 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:04.423966 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1383/2775] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 14:02:04 2022] POST /identity/v3/projects => generated 379 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 14:02:04.428010 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32909d61-92cf-4e56-a750-38059d681aac None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:04.439714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32909d61-92cf-4e56-a750-38059d681aac None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:04.494135 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881e8ae-adbe-49f0-b152-421178add656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:04.494802 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881e8ae-adbe-49f0-b152-421178add656 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:04.495087 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881e8ae-adbe-49f0-b152-421178add656 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:04.495370 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881e8ae-adbe-49f0-b152-421178add656 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:04.496069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881e8ae-adbe-49f0-b152-421178add656 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:04.498527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881e8ae-adbe-49f0-b152-421178add656 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:04.518454 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1393/2776] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 14:02:04 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 14:02:04.522450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a37974-18fd-4a3a-b56b-77ff8e199580 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:04.533249 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a37974-18fd-4a3a-b56b-77ff8e199580 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:04.537822 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ee0e2-8f71-41ef-b1c7-3cec7345c995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:04.538477 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ee0e2-8f71-41ef-b1c7-3cec7345c995 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:04.538767 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ee0e2-8f71-41ef-b1c7-3cec7345c995 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:04.539044 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ee0e2-8f71-41ef-b1c7-3cec7345c995 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:04.539765 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0ee0e2-8f71-41ef-b1c7-3cec7345c995 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:04.541427 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0ee0e2-8f71-41ef-b1c7-3cec7345c995 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:04.552109 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1384/2777] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 14:02:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 14:02:04.556216 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881e8ae-adbe-49f0-b152-421178add656 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:04.566665 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881e8ae-adbe-49f0-b152-421178add656 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:04.571266 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f9de1-a2ad-4e79-ae89-18ce63e50447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:04.572133 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f9de1-a2ad-4e79-ae89-18ce63e50447 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:04.572416 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f9de1-a2ad-4e79-ae89-18ce63e50447 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:04.572698 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f9de1-a2ad-4e79-ae89-18ce63e50447 admin admin] PATH_INFO: `/v3/projects/a66d85070fc94851bf6302a34ef3208a/users/46fbe29738e64f0ebbc4be73bf3a0d8e/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:04.584700 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:04.589344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f9de1-a2ad-4e79-ae89-18ce63e50447 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a66d85070fc94851bf6302a34ef3208a, user_id=46fbe29738e64f0ebbc4be73bf3a0d8e, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:04.593102 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f9de1-a2ad-4e79-ae89-18ce63e50447 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:04.606051 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1394/2778] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 14:02:04 2022] PUT /identity/v3/projects/a66d85070fc94851bf6302a34ef3208a/users/46fbe29738e64f0ebbc4be73bf3a0d8e/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:04.611091 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ee0e2-8f71-41ef-b1c7-3cec7345c995 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:04.620906 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ee0e2-8f71-41ef-b1c7-3cec7345c995 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:04.677396 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b10ed-e971-4799-9bcb-aecf359730a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:04.678055 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b10ed-e971-4799-9bcb-aecf359730a3 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:04.678340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b10ed-e971-4799-9bcb-aecf359730a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:04.678865 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b10ed-e971-4799-9bcb-aecf359730a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:04.679604 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b10ed-e971-4799-9bcb-aecf359730a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:04.681221 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b10ed-e971-4799-9bcb-aecf359730a3 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:04.691754 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1385/2779] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 14:02:04 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 14:02:04.695981 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f9de1-a2ad-4e79-ae89-18ce63e50447 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:04.706259 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f9de1-a2ad-4e79-ae89-18ce63e50447 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:04.711011 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b3cf1-c586-4458-a0f4-8a44689500d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:04.711890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b3cf1-c586-4458-a0f4-8a44689500d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:04.712197 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b3cf1-c586-4458-a0f4-8a44689500d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:04.712474 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b3cf1-c586-4458-a0f4-8a44689500d1 admin admin] PATH_INFO: `/v3/projects/a66d85070fc94851bf6302a34ef3208a/users/46fbe29738e64f0ebbc4be73bf3a0d8e/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:04.714470 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422b3cf1-c586-4458-a0f4-8a44689500d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a66d85070fc94851bf6302a34ef3208a, user_id=46fbe29738e64f0ebbc4be73bf3a0d8e, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:04.717344 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422b3cf1-c586-4458-a0f4-8a44689500d1 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:04.727341 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1395/2780] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 14:02:04 2022] PUT /identity/v3/projects/a66d85070fc94851bf6302a34ef3208a/users/46fbe29738e64f0ebbc4be73bf3a0d8e/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:04.737723 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57934f-a673-429f-b4ce-3436cddfc2ba None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:04.738013 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57934f-a673-429f-b4ce-3436cddfc2ba None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:04.738291 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57934f-a673-429f-b4ce-3436cddfc2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:04.764072 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-1c57934f-a673-429f-b4ce-3436cddfc2ba None None] MFA Rules not processed for user `46fbe29738e64f0ebbc4be73bf3a0d8e`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:02:04.765292 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-1c57934f-a673-429f-b4ce-3436cddfc2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:04.782754 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:04.833909 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1386/2781] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 14:02:04 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 14:02:04.845669 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b3cf1-c586-4458-a0f4-8a44689500d1 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:04.847362 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-422b3cf1-c586-4458-a0f4-8a44689500d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:04.924211 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b3cf1-c586-4458-a0f4-8a44689500d1 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:04.927830 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8571cd-05f2-4c65-92e5-30a7caa91424 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:04.928716 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8571cd-05f2-4c65-92e5-30a7caa91424 service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:04.929013 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8571cd-05f2-4c65-92e5-30a7caa91424 service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:04.929305 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8571cd-05f2-4c65-92e5-30a7caa91424 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:04.938718 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8571cd-05f2-4c65-92e5-30a7caa91424 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:04.940123 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8571cd-05f2-4c65-92e5-30a7caa91424 service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:04.949374 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1396/2782] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 14:02:04 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 14:02:05.019876 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57934f-a673-429f-b4ce-3436cddfc2ba None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:05.030176 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57934f-a673-429f-b4ce-3436cddfc2ba None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:05.033722 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add2e1c3-c064-40be-ba5c-dc6657b06a33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:05.034361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2e1c3-c064-40be-ba5c-dc6657b06a33 service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:05.034652 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2e1c3-c064-40be-ba5c-dc6657b06a33 service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:05.034929 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2e1c3-c064-40be-ba5c-dc6657b06a33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:05.090705 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add2e1c3-c064-40be-ba5c-dc6657b06a33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:05.092572 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add2e1c3-c064-40be-ba5c-dc6657b06a33 service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:05.101441 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1387/2783] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 14:02:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 14:02:14.887264 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8571cd-05f2-4c65-92e5-30a7caa91424 service cinder] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:14.949275 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8571cd-05f2-4c65-92e5-30a7caa91424 service cinder] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:14.954671 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100b85b-94d1-4c71-ab98-e735a1dd06fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:14.955555 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100b85b-94d1-4c71-ab98-e735a1dd06fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:14.955838 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100b85b-94d1-4c71-ab98-e735a1dd06fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:14.956111 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100b85b-94d1-4c71-ab98-e735a1dd06fe admin admin] PATH_INFO: `/v3/users/a47c0ebfdb7f4dc2a4da75f9016553a5` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:14.957318 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100b85b-94d1-4c71-ab98-e735a1dd06fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=a47c0ebfdb7f4dc2a4da75f9016553a5)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:14.959211 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100b85b-94d1-4c71-ab98-e735a1dd06fe admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:15.005152 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b100b85b-94d1-4c71-ab98-e735a1dd06fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a47c0ebfdb7f4dc2a4da75f9016553a5'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:15.012246 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b100b85b-94d1-4c71-ab98-e735a1dd06fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a47c0ebfdb7f4dc2a4da75f9016553a5'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:15.018455 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:15.020486 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b100b85b-94d1-4c71-ab98-e735a1dd06fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a47c0ebfdb7f4dc2a4da75f9016553a5'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:15.021381 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b100b85b-94d1-4c71-ab98-e735a1dd06fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a47c0ebfdb7f4dc2a4da75f9016553a5'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:15.035262 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-b100b85b-94d1-4c71-ab98-e735a1dd06fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a47c0ebfdb7f4dc2a4da75f9016553a5'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:15.041054 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1397/2784] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 14:02:14 2022] DELETE /identity/v3/users/a47c0ebfdb7f4dc2a4da75f9016553a5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:15.045886 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18c967-bbb1-4d02-a3ad-b6c13ccf5f3c None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:15.046386 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18c967-bbb1-4d02-a3ad-b6c13ccf5f3c None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:15.046518 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18c967-bbb1-4d02-a3ad-b6c13ccf5f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:15.064606 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-fc18c967-bbb1-4d02-a3ad-b6c13ccf5f3c None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:02:15.065823 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-fc18c967-bbb1-4d02-a3ad-b6c13ccf5f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:15.125124 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1388/2785] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 14:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 14:02:15.134628 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100b85b-94d1-4c71-ab98-e735a1dd06fe admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:15.136014 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-b100b85b-94d1-4c71-ab98-e735a1dd06fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:15.193976 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100b85b-94d1-4c71-ab98-e735a1dd06fe admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:15.197627 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1afff-3c89-4208-8a2d-9eff7a26d0d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:15.198268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1afff-3c89-4208-8a2d-9eff7a26d0d0 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:15.198550 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1afff-3c89-4208-8a2d-9eff7a26d0d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:15.198826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1afff-3c89-4208-8a2d-9eff7a26d0d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:15.208260 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1afff-3c89-4208-8a2d-9eff7a26d0d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:15.209641 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1afff-3c89-4208-8a2d-9eff7a26d0d0 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:15.219736 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1398/2786] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 14:02:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 14:02:15.486335 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18c967-bbb1-4d02-a3ad-b6c13ccf5f3c None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:15.487776 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-fc18c967-bbb1-4d02-a3ad-b6c13ccf5f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:15.499712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18c967-bbb1-4d02-a3ad-b6c13ccf5f3c None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:15.504477 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4338c40-1fe9-4c49-8cb2-8ab483f1786e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:15.505377 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4338c40-1fe9-4c49-8cb2-8ab483f1786e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:15.505664 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4338c40-1fe9-4c49-8cb2-8ab483f1786e admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:15.505939 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4338c40-1fe9-4c49-8cb2-8ab483f1786e admin admin] PATH_INFO: `/v3/projects/39cd367c83b54b17b4fff804f013f953` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:15.507118 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4338c40-1fe9-4c49-8cb2-8ab483f1786e admin admin] RBAC: Authorizing `identity:delete_project(project_id=39cd367c83b54b17b4fff804f013f953)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:15.509058 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4338c40-1fe9-4c49-8cb2-8ab483f1786e admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:15.556571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-f4338c40-1fe9-4c49-8cb2-8ab483f1786e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39cd367c83b54b17b4fff804f013f953', 'request_id': 'req-f4338c40-1fe9-4c49-8cb2-8ab483f1786e'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:15.557674 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:15.564174 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-f4338c40-1fe9-4c49-8cb2-8ab483f1786e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39cd367c83b54b17b4fff804f013f953', 'request_id': 'req-f4338c40-1fe9-4c49-8cb2-8ab483f1786e'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:15.571814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-f4338c40-1fe9-4c49-8cb2-8ab483f1786e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39cd367c83b54b17b4fff804f013f953', 'request_id': 'req-f4338c40-1fe9-4c49-8cb2-8ab483f1786e'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:15.582787 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-f4338c40-1fe9-4c49-8cb2-8ab483f1786e admin admin] The token cache is being invalidate because project 39cd367c83b54b17b4fff804f013f953 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 14:02:15.583105 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-f4338c40-1fe9-4c49-8cb2-8ab483f1786e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:15.583734 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-f4338c40-1fe9-4c49-8cb2-8ab483f1786e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:15.585395 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1389/2787] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 14:02:15 2022] DELETE /identity/v3/projects/39cd367c83b54b17b4fff804f013f953 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:15.589538 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1afff-3c89-4208-8a2d-9eff7a26d0d0 service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:15.599551 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1afff-3c89-4208-8a2d-9eff7a26d0d0 service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:15.655252 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69bd51-3396-42a6-bad1-862e3e788c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:15.656123 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69bd51-3396-42a6-bad1-862e3e788c6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:15.656418 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69bd51-3396-42a6-bad1-862e3e788c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:15.656700 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69bd51-3396-42a6-bad1-862e3e788c6b admin admin] PATH_INFO: `/v3/users/46fbe29738e64f0ebbc4be73bf3a0d8e` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:15.657927 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e69bd51-3396-42a6-bad1-862e3e788c6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=46fbe29738e64f0ebbc4be73bf3a0d8e)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:15.660112 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e69bd51-3396-42a6-bad1-862e3e788c6b admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:15.706521 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-0e69bd51-3396-42a6-bad1-862e3e788c6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46fbe29738e64f0ebbc4be73bf3a0d8e'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:15.713648 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-0e69bd51-3396-42a6-bad1-862e3e788c6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46fbe29738e64f0ebbc4be73bf3a0d8e'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:15.721091 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-0e69bd51-3396-42a6-bad1-862e3e788c6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46fbe29738e64f0ebbc4be73bf3a0d8e'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:15.721723 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-0e69bd51-3396-42a6-bad1-862e3e788c6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46fbe29738e64f0ebbc4be73bf3a0d8e'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:15.734224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-0e69bd51-3396-42a6-bad1-862e3e788c6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46fbe29738e64f0ebbc4be73bf3a0d8e'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:15.736197 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:15.740945 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1399/2788] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 14:02:15 2022] DELETE /identity/v3/users/46fbe29738e64f0ebbc4be73bf3a0d8e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:16.016583 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4338c40-1fe9-4c49-8cb2-8ab483f1786e admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:16.018245 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-f4338c40-1fe9-4c49-8cb2-8ab483f1786e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:16.034374 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4338c40-1fe9-4c49-8cb2-8ab483f1786e admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:16.090363 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1993cc-2b8a-4b72-b6b1-d0d078dacace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:16.091951 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1993cc-2b8a-4b72-b6b1-d0d078dacace admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:16.092240 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1993cc-2b8a-4b72-b6b1-d0d078dacace admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:16.092517 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1993cc-2b8a-4b72-b6b1-d0d078dacace admin admin] PATH_INFO: `/v3/projects/a66d85070fc94851bf6302a34ef3208a` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:16.093713 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1993cc-2b8a-4b72-b6b1-d0d078dacace admin admin] RBAC: Authorizing `identity:delete_project(project_id=a66d85070fc94851bf6302a34ef3208a)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:16.095610 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1993cc-2b8a-4b72-b6b1-d0d078dacace admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:16.140943 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-2f1993cc-2b8a-4b72-b6b1-d0d078dacace admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a66d85070fc94851bf6302a34ef3208a', 'request_id': 'req-2f1993cc-2b8a-4b72-b6b1-d0d078dacace'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:16.146881 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-2f1993cc-2b8a-4b72-b6b1-d0d078dacace admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a66d85070fc94851bf6302a34ef3208a', 'request_id': 'req-2f1993cc-2b8a-4b72-b6b1-d0d078dacace'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:16.156295 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-2f1993cc-2b8a-4b72-b6b1-d0d078dacace admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a66d85070fc94851bf6302a34ef3208a', 'request_id': 'req-2f1993cc-2b8a-4b72-b6b1-d0d078dacace'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:16.166782 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-2f1993cc-2b8a-4b72-b6b1-d0d078dacace admin admin] The token cache is being invalidate because project a66d85070fc94851bf6302a34ef3208a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 14:02:16.167071 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-2f1993cc-2b8a-4b72-b6b1-d0d078dacace admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:16.167686 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-2f1993cc-2b8a-4b72-b6b1-d0d078dacace admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:16.169439 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1390/2789] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 14:02:16 2022] DELETE /identity/v3/projects/a66d85070fc94851bf6302a34ef3208a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:16.174337 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89123b02-29b2-4d04-ae29-c074b8105c9c None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:16.174864 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89123b02-29b2-4d04-ae29-c074b8105c9c None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:16.174864 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89123b02-29b2-4d04-ae29-c074b8105c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:16.192135 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-89123b02-29b2-4d04-ae29-c074b8105c9c None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:02:16.193349 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-89123b02-29b2-4d04-ae29-c074b8105c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:16.260557 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1400/2790] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 14:02:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 14:02:16.268860 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be51bc-281e-40d2-8f4e-d80f418ad804 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:16.269193 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be51bc-281e-40d2-8f4e-d80f418ad804 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:16.269476 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be51bc-281e-40d2-8f4e-d80f418ad804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:16.287407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-b7be51bc-281e-40d2-8f4e-d80f418ad804 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:02:16.288603 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-b7be51bc-281e-40d2-8f4e-d80f418ad804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:16.296633 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1391/2791] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 14:02:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 14:02:16.300885 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89123b02-29b2-4d04-ae29-c074b8105c9c None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:16.311703 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89123b02-29b2-4d04-ae29-c074b8105c9c None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:16.316228 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415faee3-c44b-44cc-a308-66d2b3c3c71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:16.316876 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415faee3-c44b-44cc-a308-66d2b3c3c71d admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:16.317186 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415faee3-c44b-44cc-a308-66d2b3c3c71d admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:16.317459 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415faee3-c44b-44cc-a308-66d2b3c3c71d admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:16.318180 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415faee3-c44b-44cc-a308-66d2b3c3c71d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 14:02:16.318483 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415faee3-c44b-44cc-a308-66d2b3c3c71d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:16.320056 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415faee3-c44b-44cc-a308-66d2b3c3c71d admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:16.331966 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1401/2792] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 14:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 14:02:16.336271 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be51bc-281e-40d2-8f4e-d80f418ad804 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:16.345492 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be51bc-281e-40d2-8f4e-d80f418ad804 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:16.350115 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126eb03c-5c36-4707-8fd9-a5ba9c9127ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:16.350768 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126eb03c-5c36-4707-8fd9-a5ba9c9127ee admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:16.351050 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126eb03c-5c36-4707-8fd9-a5ba9c9127ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:16.351344 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126eb03c-5c36-4707-8fd9-a5ba9c9127ee admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:16.352009 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126eb03c-5c36-4707-8fd9-a5ba9c9127ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:16.353928 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126eb03c-5c36-4707-8fd9-a5ba9c9127ee admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:16.381122 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1392/2793] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 14:02:16 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 14:02:16.384874 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415faee3-c44b-44cc-a308-66d2b3c3c71d admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:16.395548 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415faee3-c44b-44cc-a308-66d2b3c3c71d admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:16.454039 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b727d59e-d412-40e2-a206-2531d18ed028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:16.454711 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727d59e-d412-40e2-a206-2531d18ed028 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:16.454992 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727d59e-d412-40e2-a206-2531d18ed028 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:16.455263 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727d59e-d412-40e2-a206-2531d18ed028 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:16.455961 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b727d59e-d412-40e2-a206-2531d18ed028 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:16.457937 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b727d59e-d412-40e2-a206-2531d18ed028 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:16.476807 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1402/2794] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 14:02:16 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 14:02:16.480866 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126eb03c-5c36-4707-8fd9-a5ba9c9127ee admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:16.490934 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126eb03c-5c36-4707-8fd9-a5ba9c9127ee admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:16.495449 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b07623-f3d3-4710-bff7-7f74b6666c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:16.496099 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b07623-f3d3-4710-bff7-7f74b6666c41 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:16.496382 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b07623-f3d3-4710-bff7-7f74b6666c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:16.496657 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b07623-f3d3-4710-bff7-7f74b6666c41 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:16.497930 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b07623-f3d3-4710-bff7-7f74b6666c41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:16.499500 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b07623-f3d3-4710-bff7-7f74b6666c41 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:16.510746 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1393/2795] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 14:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 14:02:16.515341 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b727d59e-d412-40e2-a206-2531d18ed028 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:16.525223 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b727d59e-d412-40e2-a206-2531d18ed028 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:16.530290 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5c8c5-9276-4a6e-8066-86ee1141c41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:16.530986 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5c8c5-9276-4a6e-8066-86ee1141c41d admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:16.531271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5c8c5-9276-4a6e-8066-86ee1141c41d admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:16.531548 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5c8c5-9276-4a6e-8066-86ee1141c41d admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:16.532326 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f5c8c5-9276-4a6e-8066-86ee1141c41d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:16.533978 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f5c8c5-9276-4a6e-8066-86ee1141c41d admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:16.544155 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1403/2796] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 14:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 14:02:16.547793 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b07623-f3d3-4710-bff7-7f74b6666c41 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:16.557982 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b07623-f3d3-4710-bff7-7f74b6666c41 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:16.563437 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2cf4d0-5732-4c20-8a63-2a1dace665c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:16.564338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2cf4d0-5732-4c20-8a63-2a1dace665c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:16.564619 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2cf4d0-5732-4c20-8a63-2a1dace665c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:16.564895 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2cf4d0-5732-4c20-8a63-2a1dace665c5 admin admin] PATH_INFO: `/v3/projects/4cba535e615b475ea4f15da7ebcfe7b0/users/61f729f6d45344718d8371dfb144605b/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:16.580400 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2cf4d0-5732-4c20-8a63-2a1dace665c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cba535e615b475ea4f15da7ebcfe7b0, user_id=61f729f6d45344718d8371dfb144605b, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:16.583189 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2cf4d0-5732-4c20-8a63-2a1dace665c5 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:16.597366 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1394/2797] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 14:02:16 2022] PUT /identity/v3/projects/4cba535e615b475ea4f15da7ebcfe7b0/users/61f729f6d45344718d8371dfb144605b/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:16.611120 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a4e72-df2f-44d9-90df-dc1ab35db4d5 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:16.611407 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a4e72-df2f-44d9-90df-dc1ab35db4d5 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:16.611689 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a4e72-df2f-44d9-90df-dc1ab35db4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:16.639096 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-f98a4e72-df2f-44d9-90df-dc1ab35db4d5 None None] MFA Rules not processed for user `61f729f6d45344718d8371dfb144605b`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:02:16.640287 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-f98a4e72-df2f-44d9-90df-dc1ab35db4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:16.695284 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:16.705288 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1404/2798] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 14:02:16 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 14:02:16.709759 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2cf4d0-5732-4c20-8a63-2a1dace665c5 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:16.715317 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:16.772724 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2cf4d0-5732-4c20-8a63-2a1dace665c5 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:16.777368 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984354b-b835-4e3f-8bef-29405b63e6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:16.778037 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984354b-b835-4e3f-8bef-29405b63e6a3 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:16.778332 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984354b-b835-4e3f-8bef-29405b63e6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:16.778748 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984354b-b835-4e3f-8bef-29405b63e6a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:16.779463 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984354b-b835-4e3f-8bef-29405b63e6a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:16.781419 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984354b-b835-4e3f-8bef-29405b63e6a3 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:16.807447 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1395/2799] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 14:02:16 2022] POST /identity/v3/projects => generated 367 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 14:02:16.812102 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a4e72-df2f-44d9-90df-dc1ab35db4d5 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:16.822826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a4e72-df2f-44d9-90df-dc1ab35db4d5 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:16.882699 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71549cb-df3f-40d7-aea0-53c754f74fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:16.883384 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71549cb-df3f-40d7-aea0-53c754f74fd6 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:16.883670 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71549cb-df3f-40d7-aea0-53c754f74fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:16.883950 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71549cb-df3f-40d7-aea0-53c754f74fd6 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:16.884694 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71549cb-df3f-40d7-aea0-53c754f74fd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:16.886747 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71549cb-df3f-40d7-aea0-53c754f74fd6 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:16.909128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1405/2800] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 14:02:16 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 14:02:16.913424 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984354b-b835-4e3f-8bef-29405b63e6a3 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:16.924220 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984354b-b835-4e3f-8bef-29405b63e6a3 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:16.929319 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f20069a-09d4-44e3-9a79-9686711e14d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:16.929985 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20069a-09d4-44e3-9a79-9686711e14d4 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:16.930274 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20069a-09d4-44e3-9a79-9686711e14d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:16.930554 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20069a-09d4-44e3-9a79-9686711e14d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:16.931287 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f20069a-09d4-44e3-9a79-9686711e14d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:16.932926 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f20069a-09d4-44e3-9a79-9686711e14d4 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:16.943388 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1396/2801] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 14:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 14:02:16.947520 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71549cb-df3f-40d7-aea0-53c754f74fd6 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:16.957895 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71549cb-df3f-40d7-aea0-53c754f74fd6 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:16.962472 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403b129-b073-4023-861a-cc3a6e30eda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:16.963347 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403b129-b073-4023-861a-cc3a6e30eda8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:16.963631 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403b129-b073-4023-861a-cc3a6e30eda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:16.963908 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403b129-b073-4023-861a-cc3a6e30eda8 admin admin] PATH_INFO: `/v3/projects/042d6e1c0ec04ec9919f5cb22833ae3a/users/01d80f3a51044507a123afe3af2e2e18/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:16.979076 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a403b129-b073-4023-861a-cc3a6e30eda8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=042d6e1c0ec04ec9919f5cb22833ae3a, user_id=01d80f3a51044507a123afe3af2e2e18, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:16.982061 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a403b129-b073-4023-861a-cc3a6e30eda8 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:16.994713 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1406/2802] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 14:02:16 2022] PUT /identity/v3/projects/042d6e1c0ec04ec9919f5cb22833ae3a/users/01d80f3a51044507a123afe3af2e2e18/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:16.999209 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f20069a-09d4-44e3-9a79-9686711e14d4 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:17.009273 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f20069a-09d4-44e3-9a79-9686711e14d4 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:17.067375 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db6fa7-d68a-46b4-ada1-f1111770b74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:17.067960 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db6fa7-d68a-46b4-ada1-f1111770b74d admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:17.068244 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db6fa7-d68a-46b4-ada1-f1111770b74d admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:17.068516 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db6fa7-d68a-46b4-ada1-f1111770b74d admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:17.069311 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db6fa7-d68a-46b4-ada1-f1111770b74d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:17.070925 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db6fa7-d68a-46b4-ada1-f1111770b74d admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:17.081285 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1397/2803] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 14:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 14:02:17.085531 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403b129-b073-4023-861a-cc3a6e30eda8 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:17.095908 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403b129-b073-4023-861a-cc3a6e30eda8 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:17.100485 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38b9f6-46cc-4367-abe7-78da311bcc0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:17.101395 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38b9f6-46cc-4367-abe7-78da311bcc0e admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:17.101683 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38b9f6-46cc-4367-abe7-78da311bcc0e admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:17.101962 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38b9f6-46cc-4367-abe7-78da311bcc0e admin admin] PATH_INFO: `/v3/projects/042d6e1c0ec04ec9919f5cb22833ae3a/users/01d80f3a51044507a123afe3af2e2e18/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:17.103968 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb38b9f6-46cc-4367-abe7-78da311bcc0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=042d6e1c0ec04ec9919f5cb22833ae3a, user_id=01d80f3a51044507a123afe3af2e2e18, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:17.106973 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb38b9f6-46cc-4367-abe7-78da311bcc0e admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:17.119498 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1407/2804] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 14:02:17 2022] PUT /identity/v3/projects/042d6e1c0ec04ec9919f5cb22833ae3a/users/01d80f3a51044507a123afe3af2e2e18/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:17.130534 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631aeb36-261c-4cb7-8bc7-3b2cec08d552 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:17.131052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631aeb36-261c-4cb7-8bc7-3b2cec08d552 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:17.131169 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631aeb36-261c-4cb7-8bc7-3b2cec08d552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:17.157518 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-631aeb36-261c-4cb7-8bc7-3b2cec08d552 None None] MFA Rules not processed for user `01d80f3a51044507a123afe3af2e2e18`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:02:17.158716 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-631aeb36-261c-4cb7-8bc7-3b2cec08d552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:17.263652 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:17.271080 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1398/2805] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 14:02:17 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 14:02:17.278462 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38b9f6-46cc-4367-abe7-78da311bcc0e admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:17.280235 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-fb38b9f6-46cc-4367-abe7-78da311bcc0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:17.337192 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38b9f6-46cc-4367-abe7-78da311bcc0e admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:17.340747 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3358f4c-c758-4dfb-b8b6-130ee3f6bad9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:17.341477 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3358f4c-c758-4dfb-b8b6-130ee3f6bad9 service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:17.341762 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3358f4c-c758-4dfb-b8b6-130ee3f6bad9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:17.342041 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3358f4c-c758-4dfb-b8b6-130ee3f6bad9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:17.350864 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3358f4c-c758-4dfb-b8b6-130ee3f6bad9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:17.352314 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3358f4c-c758-4dfb-b8b6-130ee3f6bad9 service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:17.362193 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1408/2806] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 14:02:17 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 14:02:18.681171 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631aeb36-261c-4cb7-8bc7-3b2cec08d552 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:18.746127 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631aeb36-261c-4cb7-8bc7-3b2cec08d552 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:18.751712 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32236117-cd3c-471e-9c9b-58ca1b91034d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:18.752580 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32236117-cd3c-471e-9c9b-58ca1b91034d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:18.752863 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32236117-cd3c-471e-9c9b-58ca1b91034d admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:18.753168 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32236117-cd3c-471e-9c9b-58ca1b91034d admin admin] PATH_INFO: `/v3/users/61f729f6d45344718d8371dfb144605b` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:18.754347 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32236117-cd3c-471e-9c9b-58ca1b91034d admin admin] RBAC: Authorizing `identity:delete_user(user_id=61f729f6d45344718d8371dfb144605b)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:18.756259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32236117-cd3c-471e-9c9b-58ca1b91034d admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:18.800200 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-32236117-cd3c-471e-9c9b-58ca1b91034d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61f729f6d45344718d8371dfb144605b'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:18.808618 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-32236117-cd3c-471e-9c9b-58ca1b91034d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61f729f6d45344718d8371dfb144605b'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:18.809894 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-32236117-cd3c-471e-9c9b-58ca1b91034d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61f729f6d45344718d8371dfb144605b'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:18.817265 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-32236117-cd3c-471e-9c9b-58ca1b91034d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61f729f6d45344718d8371dfb144605b'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:18.830078 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-32236117-cd3c-471e-9c9b-58ca1b91034d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61f729f6d45344718d8371dfb144605b'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:18.836049 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1399/2807] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 14:02:18 2022] DELETE /identity/v3/users/61f729f6d45344718d8371dfb144605b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:18.840552 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b72ae8-a4b4-4af3-9abb-638808baf975 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:18.841170 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b72ae8-a4b4-4af3-9abb-638808baf975 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:18.841291 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b72ae8-a4b4-4af3-9abb-638808baf975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:18.859131 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-94b72ae8-a4b4-4af3-9abb-638808baf975 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:02:18.860236 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-94b72ae8-a4b4-4af3-9abb-638808baf975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:18.902603 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:18.917507 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1409/2808] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 14:02:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 14:02:18.927790 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32236117-cd3c-471e-9c9b-58ca1b91034d admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:18.929521 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-32236117-cd3c-471e-9c9b-58ca1b91034d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:18.990427 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32236117-cd3c-471e-9c9b-58ca1b91034d admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:18.994055 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538466ce-7aae-43ea-99de-3cee8aa57fb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:18.994708 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538466ce-7aae-43ea-99de-3cee8aa57fb1 service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:18.994991 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538466ce-7aae-43ea-99de-3cee8aa57fb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:18.995269 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538466ce-7aae-43ea-99de-3cee8aa57fb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:19.004877 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538466ce-7aae-43ea-99de-3cee8aa57fb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:19.006343 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538466ce-7aae-43ea-99de-3cee8aa57fb1 service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:19.015874 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1400/2809] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 14:02:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 14:02:19.303854 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b72ae8-a4b4-4af3-9abb-638808baf975 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:19.305317 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-94b72ae8-a4b4-4af3-9abb-638808baf975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:19.317663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b72ae8-a4b4-4af3-9abb-638808baf975 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:19.322235 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7b33e-a9b8-4023-b0b1-ec449f6bf420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:19.323097 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7b33e-a9b8-4023-b0b1-ec449f6bf420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:19.323565 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7b33e-a9b8-4023-b0b1-ec449f6bf420 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:19.323763 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7b33e-a9b8-4023-b0b1-ec449f6bf420 admin admin] PATH_INFO: `/v3/projects/4cba535e615b475ea4f15da7ebcfe7b0` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:19.324953 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e7b33e-a9b8-4023-b0b1-ec449f6bf420 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cba535e615b475ea4f15da7ebcfe7b0)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:19.326872 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e7b33e-a9b8-4023-b0b1-ec449f6bf420 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:19.372906 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-d8e7b33e-a9b8-4023-b0b1-ec449f6bf420 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cba535e615b475ea4f15da7ebcfe7b0', 'request_id': 'req-d8e7b33e-a9b8-4023-b0b1-ec449f6bf420'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:19.378309 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-d8e7b33e-a9b8-4023-b0b1-ec449f6bf420 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cba535e615b475ea4f15da7ebcfe7b0', 'request_id': 'req-d8e7b33e-a9b8-4023-b0b1-ec449f6bf420'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:19.385824 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-d8e7b33e-a9b8-4023-b0b1-ec449f6bf420 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cba535e615b475ea4f15da7ebcfe7b0', 'request_id': 'req-d8e7b33e-a9b8-4023-b0b1-ec449f6bf420'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:19.397267 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-d8e7b33e-a9b8-4023-b0b1-ec449f6bf420 admin admin] The token cache is being invalidate because project 4cba535e615b475ea4f15da7ebcfe7b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 14:02:19.397562 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-d8e7b33e-a9b8-4023-b0b1-ec449f6bf420 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:19.398211 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-d8e7b33e-a9b8-4023-b0b1-ec449f6bf420 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:19.399745 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1410/2810] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 14:02:19 2022] DELETE /identity/v3/projects/4cba535e615b475ea4f15da7ebcfe7b0 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:19.404257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538466ce-7aae-43ea-99de-3cee8aa57fb1 service neutron] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:19.413908 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538466ce-7aae-43ea-99de-3cee8aa57fb1 service neutron] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:19.432678 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:19.471684 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05c38d3-85c6-4f22-8f27-a5ff929a7afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:19.472544 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c38d3-85c6-4f22-8f27-a5ff929a7afa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:19.472832 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c38d3-85c6-4f22-8f27-a5ff929a7afa admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:19.473151 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c38d3-85c6-4f22-8f27-a5ff929a7afa admin admin] PATH_INFO: `/v3/users/01d80f3a51044507a123afe3af2e2e18` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:19.474351 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05c38d3-85c6-4f22-8f27-a5ff929a7afa admin admin] RBAC: Authorizing `identity:delete_user(user_id=01d80f3a51044507a123afe3af2e2e18)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:19.476265 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05c38d3-85c6-4f22-8f27-a5ff929a7afa admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:19.519147 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-c05c38d3-85c6-4f22-8f27-a5ff929a7afa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01d80f3a51044507a123afe3af2e2e18'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:19.525735 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-c05c38d3-85c6-4f22-8f27-a5ff929a7afa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01d80f3a51044507a123afe3af2e2e18'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:19.526346 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-c05c38d3-85c6-4f22-8f27-a5ff929a7afa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01d80f3a51044507a123afe3af2e2e18'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:19.534014 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-c05c38d3-85c6-4f22-8f27-a5ff929a7afa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01d80f3a51044507a123afe3af2e2e18'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:19.546497 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-c05c38d3-85c6-4f22-8f27-a5ff929a7afa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01d80f3a51044507a123afe3af2e2e18'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:19.552269 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1401/2811] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 14:02:19 2022] DELETE /identity/v3/users/01d80f3a51044507a123afe3af2e2e18 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:19.825729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7b33e-a9b8-4023-b0b1-ec449f6bf420 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:19.827714 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-d8e7b33e-a9b8-4023-b0b1-ec449f6bf420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:19.841415 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7b33e-a9b8-4023-b0b1-ec449f6bf420 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:19.898540 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1905c2-0a6a-4bc4-aab3-e8a111c0e5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:19.899399 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1905c2-0a6a-4bc4-aab3-e8a111c0e5e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:19.899683 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1905c2-0a6a-4bc4-aab3-e8a111c0e5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:19.899960 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1905c2-0a6a-4bc4-aab3-e8a111c0e5e1 admin admin] PATH_INFO: `/v3/projects/042d6e1c0ec04ec9919f5cb22833ae3a` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:19.901203 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1905c2-0a6a-4bc4-aab3-e8a111c0e5e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=042d6e1c0ec04ec9919f5cb22833ae3a)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:19.904295 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1905c2-0a6a-4bc4-aab3-e8a111c0e5e1 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:19.949576 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-bf1905c2-0a6a-4bc4-aab3-e8a111c0e5e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '042d6e1c0ec04ec9919f5cb22833ae3a', 'request_id': 'req-bf1905c2-0a6a-4bc4-aab3-e8a111c0e5e1'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:19.955447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-bf1905c2-0a6a-4bc4-aab3-e8a111c0e5e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '042d6e1c0ec04ec9919f5cb22833ae3a', 'request_id': 'req-bf1905c2-0a6a-4bc4-aab3-e8a111c0e5e1'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:19.963407 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-bf1905c2-0a6a-4bc4-aab3-e8a111c0e5e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '042d6e1c0ec04ec9919f5cb22833ae3a', 'request_id': 'req-bf1905c2-0a6a-4bc4-aab3-e8a111c0e5e1'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:19.975276 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-bf1905c2-0a6a-4bc4-aab3-e8a111c0e5e1 admin admin] The token cache is being invalidate because project 042d6e1c0ec04ec9919f5cb22833ae3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 14:02:19.975563 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-bf1905c2-0a6a-4bc4-aab3-e8a111c0e5e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:19.976192 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-bf1905c2-0a6a-4bc4-aab3-e8a111c0e5e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:19.977734 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1411/2812] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 14:02:19 2022] DELETE /identity/v3/projects/042d6e1c0ec04ec9919f5cb22833ae3a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 14:02:19.982741 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95dcaa-d2e9-4539-b411-2793dccb0563 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:19.983269 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95dcaa-d2e9-4539-b411-2793dccb0563 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:19.983377 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95dcaa-d2e9-4539-b411-2793dccb0563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:20.003439 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-dd95dcaa-d2e9-4539-b411-2793dccb0563 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:02:20.004624 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-dd95dcaa-d2e9-4539-b411-2793dccb0563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:20.063569 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1402/2813] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 14:02:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 14:02:20.072165 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee863a-258b-4e73-b707-670fda0047d5 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:20.072474 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee863a-258b-4e73-b707-670fda0047d5 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:20.072758 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee863a-258b-4e73-b707-670fda0047d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:20.080658 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:20.092189 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-f8ee863a-258b-4e73-b707-670fda0047d5 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:02:20.093413 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-f8ee863a-258b-4e73-b707-670fda0047d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:20.101221 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1412/2814] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 14:02:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 14:02:20.105486 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95dcaa-d2e9-4539-b411-2793dccb0563 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:20.115462 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95dcaa-d2e9-4539-b411-2793dccb0563 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:20.120082 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691209b-2385-45dd-9d87-7e9a1f6e8403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:20.120773 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691209b-2385-45dd-9d87-7e9a1f6e8403 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:20.121102 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691209b-2385-45dd-9d87-7e9a1f6e8403 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:20.121384 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691209b-2385-45dd-9d87-7e9a1f6e8403 admin admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:20.122682 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a691209b-2385-45dd-9d87-7e9a1f6e8403 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 14:02:20.122990 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a691209b-2385-45dd-9d87-7e9a1f6e8403 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:20.124626 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a691209b-2385-45dd-9d87-7e9a1f6e8403 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:20.136037 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1403/2815] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 14:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 14:02:20.140777 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee863a-258b-4e73-b707-670fda0047d5 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:20.149882 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee863a-258b-4e73-b707-670fda0047d5 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:20.154816 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5c716-94ea-4053-aa5f-9b824805a915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:20.155399 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5c716-94ea-4053-aa5f-9b824805a915 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:20.155686 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5c716-94ea-4053-aa5f-9b824805a915 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:20.155961 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5c716-94ea-4053-aa5f-9b824805a915 admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:20.156664 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5c716-94ea-4053-aa5f-9b824805a915 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:20.158575 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5c716-94ea-4053-aa5f-9b824805a915 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:20.183301 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1413/2816] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 14:02:20 2022] POST /identity/v3/projects => generated 367 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 14:02:20.187980 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691209b-2385-45dd-9d87-7e9a1f6e8403 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:20.197956 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691209b-2385-45dd-9d87-7e9a1f6e8403 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:20.223861 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:20.256896 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede196c2-114a-428c-8b68-b67c018d6919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:20.257595 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede196c2-114a-428c-8b68-b67c018d6919 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:20.257890 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede196c2-114a-428c-8b68-b67c018d6919 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:20.258169 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede196c2-114a-428c-8b68-b67c018d6919 admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:20.258865 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede196c2-114a-428c-8b68-b67c018d6919 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:20.260827 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede196c2-114a-428c-8b68-b67c018d6919 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:20.280468 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1404/2817] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 14:02:20 2022] POST /identity/v3/users => generated 284 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 14:02:20.284487 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5c716-94ea-4053-aa5f-9b824805a915 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:20.295069 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5c716-94ea-4053-aa5f-9b824805a915 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:20.299811 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e43da-fdbb-4a7b-b9f7-aa0953b623e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:20.300492 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e43da-fdbb-4a7b-b9f7-aa0953b623e6 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:20.300756 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e43da-fdbb-4a7b-b9f7-aa0953b623e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:20.301296 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e43da-fdbb-4a7b-b9f7-aa0953b623e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:20.302030 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e43da-fdbb-4a7b-b9f7-aa0953b623e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:20.303860 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e43da-fdbb-4a7b-b9f7-aa0953b623e6 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:20.314945 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1414/2818] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 14:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 14:02:20.318915 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede196c2-114a-428c-8b68-b67c018d6919 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:20.329584 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede196c2-114a-428c-8b68-b67c018d6919 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:20.334167 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee3b17-29e7-442a-9c75-97b407532890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:20.334825 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee3b17-29e7-442a-9c75-97b407532890 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:20.335112 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee3b17-29e7-442a-9c75-97b407532890 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:20.335417 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee3b17-29e7-442a-9c75-97b407532890 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:20.336273 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bee3b17-29e7-442a-9c75-97b407532890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:20.337904 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bee3b17-29e7-442a-9c75-97b407532890 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:20.347862 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1405/2819] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 14:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 14:02:20.351802 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e43da-fdbb-4a7b-b9f7-aa0953b623e6 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:20.362388 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e43da-fdbb-4a7b-b9f7-aa0953b623e6 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:20.366908 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ee16f-7240-433e-87db-811dab9525e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:20.367800 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ee16f-7240-433e-87db-811dab9525e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:20.368080 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ee16f-7240-433e-87db-811dab9525e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:20.368355 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ee16f-7240-433e-87db-811dab9525e6 admin admin] PATH_INFO: `/v3/projects/e677b4d6064847e2a57d202673402d41/users/ffa9eb9f1b274b579e18a8ebb3b9ec67/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:20.384161 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81ee16f-7240-433e-87db-811dab9525e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e677b4d6064847e2a57d202673402d41, user_id=ffa9eb9f1b274b579e18a8ebb3b9ec67, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:20.386810 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81ee16f-7240-433e-87db-811dab9525e6 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:20.400019 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1415/2820] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 14:02:20 2022] PUT /identity/v3/projects/e677b4d6064847e2a57d202673402d41/users/ffa9eb9f1b274b579e18a8ebb3b9ec67/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:20.460064 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7934c-bc05-415c-b558-469d03362e80 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:20.460295 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7934c-bc05-415c-b558-469d03362e80 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:20.460543 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7934c-bc05-415c-b558-469d03362e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:20.486976 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-6de7934c-bc05-415c-b558-469d03362e80 None None] MFA Rules not processed for user `ffa9eb9f1b274b579e18a8ebb3b9ec67`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:02:20.488187 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-6de7934c-bc05-415c-b558-469d03362e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:20.550263 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1406/2821] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 14:02:20 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 14:02:20.555098 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ee16f-7240-433e-87db-811dab9525e6 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:20.615973 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ee16f-7240-433e-87db-811dab9525e6 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:20.620555 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78898d3-2820-4604-857e-5978c9eda08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:20.621240 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78898d3-2820-4604-857e-5978c9eda08d admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:20.621527 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78898d3-2820-4604-857e-5978c9eda08d admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:20.622528 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78898d3-2820-4604-857e-5978c9eda08d admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:20.623217 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78898d3-2820-4604-857e-5978c9eda08d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:20.625129 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78898d3-2820-4604-857e-5978c9eda08d admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:20.649426 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1416/2822] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 14:02:20 2022] POST /identity/v3/projects => generated 367 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 14:02:20.653495 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de7934c-bc05-415c-b558-469d03362e80 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:20.664319 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de7934c-bc05-415c-b558-469d03362e80 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:20.721579 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9acaab1-0cfb-48c3-8afe-93bd35ca5f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:20.722180 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9acaab1-0cfb-48c3-8afe-93bd35ca5f32 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:20.722471 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9acaab1-0cfb-48c3-8afe-93bd35ca5f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:20.722748 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9acaab1-0cfb-48c3-8afe-93bd35ca5f32 admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:20.723483 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9acaab1-0cfb-48c3-8afe-93bd35ca5f32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:20.725471 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9acaab1-0cfb-48c3-8afe-93bd35ca5f32 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:20.744873 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1407/2823] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 14:02:20 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 14:02:20.749184 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78898d3-2820-4604-857e-5978c9eda08d admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:20.759175 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78898d3-2820-4604-857e-5978c9eda08d admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:20.763962 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782eff54-3a49-4865-8b16-d8dcd4d46640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:20.764613 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782eff54-3a49-4865-8b16-d8dcd4d46640 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:20.764894 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782eff54-3a49-4865-8b16-d8dcd4d46640 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:20.765200 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782eff54-3a49-4865-8b16-d8dcd4d46640 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:20.765928 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782eff54-3a49-4865-8b16-d8dcd4d46640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:20.767508 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782eff54-3a49-4865-8b16-d8dcd4d46640 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:20.777802 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1417/2824] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 14:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 14:02:20.782270 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9acaab1-0cfb-48c3-8afe-93bd35ca5f32 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:20.791932 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9acaab1-0cfb-48c3-8afe-93bd35ca5f32 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:20.797143 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4cc6c-e3be-4f9e-9cb4-05ec8cc7d38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:20.798015 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4cc6c-e3be-4f9e-9cb4-05ec8cc7d38c admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:20.798293 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4cc6c-e3be-4f9e-9cb4-05ec8cc7d38c admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:20.798565 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4cc6c-e3be-4f9e-9cb4-05ec8cc7d38c admin admin] PATH_INFO: `/v3/projects/3750d508fa9c458ca98f2a7e1a964ebc/users/ff235069b95e4aed82e7a21f7fd011ec/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:20.812974 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc4cc6c-e3be-4f9e-9cb4-05ec8cc7d38c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3750d508fa9c458ca98f2a7e1a964ebc, user_id=ff235069b95e4aed82e7a21f7fd011ec, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:20.815611 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc4cc6c-e3be-4f9e-9cb4-05ec8cc7d38c admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:20.828870 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1408/2825] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 14:02:20 2022] PUT /identity/v3/projects/3750d508fa9c458ca98f2a7e1a964ebc/users/ff235069b95e4aed82e7a21f7fd011ec/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:20.832692 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782eff54-3a49-4865-8b16-d8dcd4d46640 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:20.843320 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782eff54-3a49-4865-8b16-d8dcd4d46640 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:20.897727 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c8301-9bb6-474e-8daa-d657f3833395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:20.898382 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c8301-9bb6-474e-8daa-d657f3833395 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:20.898667 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c8301-9bb6-474e-8daa-d657f3833395 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:20.898945 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c8301-9bb6-474e-8daa-d657f3833395 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:20.899661 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9c8301-9bb6-474e-8daa-d657f3833395 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:20.901261 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9c8301-9bb6-474e-8daa-d657f3833395 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:20.911316 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1418/2826] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 14:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 14:02:20.915409 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4cc6c-e3be-4f9e-9cb4-05ec8cc7d38c admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:20.921637 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:20.927875 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4cc6c-e3be-4f9e-9cb4-05ec8cc7d38c admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:20.933145 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82afd8-7821-4e9c-b0ff-5863c9f208f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:20.934026 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82afd8-7821-4e9c-b0ff-5863c9f208f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:20.934456 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82afd8-7821-4e9c-b0ff-5863c9f208f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:20.934674 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82afd8-7821-4e9c-b0ff-5863c9f208f7 admin admin] PATH_INFO: `/v3/projects/3750d508fa9c458ca98f2a7e1a964ebc/users/ff235069b95e4aed82e7a21f7fd011ec/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:20.936665 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c82afd8-7821-4e9c-b0ff-5863c9f208f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3750d508fa9c458ca98f2a7e1a964ebc, user_id=ff235069b95e4aed82e7a21f7fd011ec, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:20.939338 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c82afd8-7821-4e9c-b0ff-5863c9f208f7 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:20.949419 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1409/2827] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 14:02:20 2022] PUT /identity/v3/projects/3750d508fa9c458ca98f2a7e1a964ebc/users/ff235069b95e4aed82e7a21f7fd011ec/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:20.959264 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2622c-e483-4395-bf2f-3b9823e522b1 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:20.959559 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2622c-e483-4395-bf2f-3b9823e522b1 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:20.959841 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2622c-e483-4395-bf2f-3b9823e522b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:20.979587 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:20.988296 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-38e2622c-e483-4395-bf2f-3b9823e522b1 None None] MFA Rules not processed for user `ff235069b95e4aed82e7a21f7fd011ec`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:02:20.989718 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-38e2622c-e483-4395-bf2f-3b9823e522b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:21.056687 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1419/2828] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 14:02:20 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 14:02:21.067931 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82afd8-7821-4e9c-b0ff-5863c9f208f7 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:21.069646 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-3c82afd8-7821-4e9c-b0ff-5863c9f208f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:21.127838 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82afd8-7821-4e9c-b0ff-5863c9f208f7 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:21.131474 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f01e01-140c-4fae-899b-d91c944497c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:21.132115 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f01e01-140c-4fae-899b-d91c944497c6 service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:21.132405 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f01e01-140c-4fae-899b-d91c944497c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:21.132684 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f01e01-140c-4fae-899b-d91c944497c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:21.144266 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f01e01-140c-4fae-899b-d91c944497c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:21.146295 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f01e01-140c-4fae-899b-d91c944497c6 service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:21.157864 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1410/2829] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 14:02:21 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 14:02:21.199491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e2622c-e483-4395-bf2f-3b9823e522b1 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:21.201561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-38e2622c-e483-4395-bf2f-3b9823e522b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:21.213311 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e2622c-e483-4395-bf2f-3b9823e522b1 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:21.217519 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa47a7-3da2-47e7-9c48-1f31c2bb40a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:21.218148 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa47a7-3da2-47e7-9c48-1f31c2bb40a7 service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:21.218427 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa47a7-3da2-47e7-9c48-1f31c2bb40a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:21.218699 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa47a7-3da2-47e7-9c48-1f31c2bb40a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:21.273705 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa47a7-3da2-47e7-9c48-1f31c2bb40a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:21.275128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa47a7-3da2-47e7-9c48-1f31c2bb40a7 service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:21.284557 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1420/2830] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 14:02:21 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 14:02:24.548053 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f01e01-140c-4fae-899b-d91c944497c6 service cinder] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:24.608834 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f01e01-140c-4fae-899b-d91c944497c6 service cinder] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:24.613390 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b934fd-8a30-4293-92e4-ae917307098a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:24.614245 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b934fd-8a30-4293-92e4-ae917307098a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:24.614523 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b934fd-8a30-4293-92e4-ae917307098a admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:24.614795 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b934fd-8a30-4293-92e4-ae917307098a admin admin] PATH_INFO: `/v3/users/ffa9eb9f1b274b579e18a8ebb3b9ec67` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:24.616214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b934fd-8a30-4293-92e4-ae917307098a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffa9eb9f1b274b579e18a8ebb3b9ec67)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:24.618172 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b934fd-8a30-4293-92e4-ae917307098a admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:24.663354 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-b1b934fd-8a30-4293-92e4-ae917307098a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffa9eb9f1b274b579e18a8ebb3b9ec67'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:24.671939 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-b1b934fd-8a30-4293-92e4-ae917307098a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffa9eb9f1b274b579e18a8ebb3b9ec67'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:24.672595 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-b1b934fd-8a30-4293-92e4-ae917307098a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffa9eb9f1b274b579e18a8ebb3b9ec67'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:24.679676 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-b1b934fd-8a30-4293-92e4-ae917307098a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffa9eb9f1b274b579e18a8ebb3b9ec67'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:24.691890 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-b1b934fd-8a30-4293-92e4-ae917307098a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffa9eb9f1b274b579e18a8ebb3b9ec67'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:24.697703 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1411/2831] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 14:02:24 2022] DELETE /identity/v3/users/ffa9eb9f1b274b579e18a8ebb3b9ec67 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:24.703358 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883bc9f6-e7e7-4214-bf03-90f185f6d5b8 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:24.703849 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883bc9f6-e7e7-4214-bf03-90f185f6d5b8 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:24.703956 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883bc9f6-e7e7-4214-bf03-90f185f6d5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:24.721086 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-883bc9f6-e7e7-4214-bf03-90f185f6d5b8 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:02:24.722292 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-883bc9f6-e7e7-4214-bf03-90f185f6d5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:24.781766 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1421/2832] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 14:02:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 14:02:24.791222 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b934fd-8a30-4293-92e4-ae917307098a admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:24.792636 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-b1b934fd-8a30-4293-92e4-ae917307098a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:24.806378 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:24.854339 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b934fd-8a30-4293-92e4-ae917307098a admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:24.858501 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d86db-c2c6-4429-a3a9-49934f31156b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:24.859142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d86db-c2c6-4429-a3a9-49934f31156b service neutron] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:24.859426 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d86db-c2c6-4429-a3a9-49934f31156b service neutron] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:24.859702 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d86db-c2c6-4429-a3a9-49934f31156b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:24.868602 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066d86db-c2c6-4429-a3a9-49934f31156b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:24.870055 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066d86db-c2c6-4429-a3a9-49934f31156b service neutron] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:24.880559 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1412/2833] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 14:02:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 14:02:25.149144 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883bc9f6-e7e7-4214-bf03-90f185f6d5b8 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:25.150709 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-883bc9f6-e7e7-4214-bf03-90f185f6d5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:25.164319 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883bc9f6-e7e7-4214-bf03-90f185f6d5b8 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:25.168901 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f928289-b108-4e1d-a38e-cf0959e8d6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:25.170351 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f928289-b108-4e1d-a38e-cf0959e8d6cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:25.170643 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f928289-b108-4e1d-a38e-cf0959e8d6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:25.170920 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f928289-b108-4e1d-a38e-cf0959e8d6cf admin admin] PATH_INFO: `/v3/projects/e677b4d6064847e2a57d202673402d41` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:25.172117 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f928289-b108-4e1d-a38e-cf0959e8d6cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=e677b4d6064847e2a57d202673402d41)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:25.174108 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f928289-b108-4e1d-a38e-cf0959e8d6cf admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:25.209720 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:25.225551 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-3f928289-b108-4e1d-a38e-cf0959e8d6cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e677b4d6064847e2a57d202673402d41', 'request_id': 'req-3f928289-b108-4e1d-a38e-cf0959e8d6cf'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:25.231372 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-3f928289-b108-4e1d-a38e-cf0959e8d6cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e677b4d6064847e2a57d202673402d41', 'request_id': 'req-3f928289-b108-4e1d-a38e-cf0959e8d6cf'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:25.238935 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-3f928289-b108-4e1d-a38e-cf0959e8d6cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e677b4d6064847e2a57d202673402d41', 'request_id': 'req-3f928289-b108-4e1d-a38e-cf0959e8d6cf'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:25.250219 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-3f928289-b108-4e1d-a38e-cf0959e8d6cf admin admin] The token cache is being invalidate because project e677b4d6064847e2a57d202673402d41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 14:02:25.250504 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-3f928289-b108-4e1d-a38e-cf0959e8d6cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:25.251120 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-3f928289-b108-4e1d-a38e-cf0959e8d6cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:25.252649 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1422/2834] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 14:02:25 2022] DELETE /identity/v3/projects/e677b4d6064847e2a57d202673402d41 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:25.256525 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d86db-c2c6-4429-a3a9-49934f31156b service neutron] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:25.266348 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d86db-c2c6-4429-a3a9-49934f31156b service neutron] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:25.322255 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdb5ef-2313-4aa0-898e-edafbb299ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:25.323117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdb5ef-2313-4aa0-898e-edafbb299ff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:25.323398 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdb5ef-2313-4aa0-898e-edafbb299ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:25.323671 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdb5ef-2313-4aa0-898e-edafbb299ff9 admin admin] PATH_INFO: `/v3/users/ff235069b95e4aed82e7a21f7fd011ec` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:25.324826 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdb5ef-2313-4aa0-898e-edafbb299ff9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff235069b95e4aed82e7a21f7fd011ec)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:25.327031 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdb5ef-2313-4aa0-898e-edafbb299ff9 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:25.373539 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-68cdb5ef-2313-4aa0-898e-edafbb299ff9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff235069b95e4aed82e7a21f7fd011ec'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:25.380889 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-68cdb5ef-2313-4aa0-898e-edafbb299ff9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff235069b95e4aed82e7a21f7fd011ec'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:25.381541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-68cdb5ef-2313-4aa0-898e-edafbb299ff9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff235069b95e4aed82e7a21f7fd011ec'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:25.388454 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-68cdb5ef-2313-4aa0-898e-edafbb299ff9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff235069b95e4aed82e7a21f7fd011ec'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:25.400766 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-68cdb5ef-2313-4aa0-898e-edafbb299ff9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff235069b95e4aed82e7a21f7fd011ec'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:25.407663 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1413/2835] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 14:02:25 2022] DELETE /identity/v3/users/ff235069b95e4aed82e7a21f7fd011ec => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:25.668260 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f928289-b108-4e1d-a38e-cf0959e8d6cf admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:25.669804 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-3f928289-b108-4e1d-a38e-cf0959e8d6cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:25.682646 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f928289-b108-4e1d-a38e-cf0959e8d6cf admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:25.740846 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c22b9-c3fd-4604-8f31-982902696a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:25.741736 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c22b9-c3fd-4604-8f31-982902696a44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:25.742019 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c22b9-c3fd-4604-8f31-982902696a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:25.742293 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c22b9-c3fd-4604-8f31-982902696a44 admin admin] PATH_INFO: `/v3/projects/3750d508fa9c458ca98f2a7e1a964ebc` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:25.743466 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9c22b9-c3fd-4604-8f31-982902696a44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3750d508fa9c458ca98f2a7e1a964ebc)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:25.745392 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9c22b9-c3fd-4604-8f31-982902696a44 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:25.790609 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-9d9c22b9-c3fd-4604-8f31-982902696a44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3750d508fa9c458ca98f2a7e1a964ebc', 'request_id': 'req-9d9c22b9-c3fd-4604-8f31-982902696a44'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:25.796184 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-9d9c22b9-c3fd-4604-8f31-982902696a44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3750d508fa9c458ca98f2a7e1a964ebc', 'request_id': 'req-9d9c22b9-c3fd-4604-8f31-982902696a44'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:25.803263 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-9d9c22b9-c3fd-4604-8f31-982902696a44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3750d508fa9c458ca98f2a7e1a964ebc', 'request_id': 'req-9d9c22b9-c3fd-4604-8f31-982902696a44'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:25.814473 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-9d9c22b9-c3fd-4604-8f31-982902696a44 admin admin] The token cache is being invalidate because project 3750d508fa9c458ca98f2a7e1a964ebc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86849) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 14:02:25.814759 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-9d9c22b9-c3fd-4604-8f31-982902696a44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:25.815377 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-9d9c22b9-c3fd-4604-8f31-982902696a44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:25.816898 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1423/2836] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 14:02:25 2022] DELETE /identity/v3/projects/3750d508fa9c458ca98f2a7e1a964ebc => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:25.821900 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce924d0-9ca3-44b2-b1a9-fb1ec7ba88f7 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:25.822382 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce924d0-9ca3-44b2-b1a9-fb1ec7ba88f7 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:25.822526 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce924d0-9ca3-44b2-b1a9-fb1ec7ba88f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:25.841122 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-1ce924d0-9ca3-44b2-b1a9-fb1ec7ba88f7 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:02:25.843026 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-1ce924d0-9ca3-44b2-b1a9-fb1ec7ba88f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:25.871568 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:25.903175 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1414/2837] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 14:02:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 14:02:25.912334 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9d0ec-2cb4-40f7-bded-1bb52375fe15 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:25.912618 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9d0ec-2cb4-40f7-bded-1bb52375fe15 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:25.912921 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9d0ec-2cb4-40f7-bded-1bb52375fe15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:25.931125 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-4ea9d0ec-2cb4-40f7-bded-1bb52375fe15 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:02:25.932305 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-4ea9d0ec-2cb4-40f7-bded-1bb52375fe15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:25.940648 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1424/2838] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 14:02:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 14:02:25.944749 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce924d0-9ca3-44b2-b1a9-fb1ec7ba88f7 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:25.955177 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce924d0-9ca3-44b2-b1a9-fb1ec7ba88f7 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:25.960038 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23ac0d-04d3-44db-9084-e05020401517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:25.960693 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23ac0d-04d3-44db-9084-e05020401517 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:25.961069 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23ac0d-04d3-44db-9084-e05020401517 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:25.961345 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23ac0d-04d3-44db-9084-e05020401517 admin admin] PATH_INFO: `/v3/domains` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:25.962073 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23ac0d-04d3-44db-9084-e05020401517 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86850) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 14:02:25.962400 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23ac0d-04d3-44db-9084-e05020401517 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:25.964015 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23ac0d-04d3-44db-9084-e05020401517 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:25.975546 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1415/2839] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 14:02:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 14:02:25.979687 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea9d0ec-2cb4-40f7-bded-1bb52375fe15 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:25.988783 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea9d0ec-2cb4-40f7-bded-1bb52375fe15 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:25.993424 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6730931-5aa9-4736-9ea7-46ca007c5c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:25.994061 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6730931-5aa9-4736-9ea7-46ca007c5c93 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:25.994345 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6730931-5aa9-4736-9ea7-46ca007c5c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:25.994620 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6730931-5aa9-4736-9ea7-46ca007c5c93 admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:25.995316 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6730931-5aa9-4736-9ea7-46ca007c5c93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:25.997185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6730931-5aa9-4736-9ea7-46ca007c5c93 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:26.024556 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1425/2840] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 14:02:25 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 14:02:26.028582 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23ac0d-04d3-44db-9084-e05020401517 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:26.039476 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23ac0d-04d3-44db-9084-e05020401517 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:26.095094 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c6e438-717d-4f74-a031-bebb4a826a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:26.095659 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6e438-717d-4f74-a031-bebb4a826a46 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:26.095946 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6e438-717d-4f74-a031-bebb4a826a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:26.096219 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6e438-717d-4f74-a031-bebb4a826a46 admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:26.096901 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6e438-717d-4f74-a031-bebb4a826a46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:26.098816 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6e438-717d-4f74-a031-bebb4a826a46 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:26.118793 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1416/2841] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 14:02:26 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 14:02:26.123337 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6730931-5aa9-4736-9ea7-46ca007c5c93 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:26.133366 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6730931-5aa9-4736-9ea7-46ca007c5c93 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:26.137888 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc989170-d6d6-4cbc-852d-9cd514bf9404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:26.139177 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc989170-d6d6-4cbc-852d-9cd514bf9404 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:26.139455 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc989170-d6d6-4cbc-852d-9cd514bf9404 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:26.139729 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc989170-d6d6-4cbc-852d-9cd514bf9404 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:26.140447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc989170-d6d6-4cbc-852d-9cd514bf9404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:26.142049 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc989170-d6d6-4cbc-852d-9cd514bf9404 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:26.152177 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1426/2842] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 14:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 14:02:26.156032 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c6e438-717d-4f74-a031-bebb4a826a46 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:26.166167 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c6e438-717d-4f74-a031-bebb4a826a46 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:26.170857 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e34b5-1172-4629-961e-2b4f1c30c536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:26.171503 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e34b5-1172-4629-961e-2b4f1c30c536 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:26.171785 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e34b5-1172-4629-961e-2b4f1c30c536 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:26.172058 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e34b5-1172-4629-961e-2b4f1c30c536 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:26.172803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67e34b5-1172-4629-961e-2b4f1c30c536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:26.174407 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67e34b5-1172-4629-961e-2b4f1c30c536 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:26.184245 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1417/2843] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 14:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 14:02:26.188826 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc989170-d6d6-4cbc-852d-9cd514bf9404 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:26.198459 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc989170-d6d6-4cbc-852d-9cd514bf9404 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:26.203604 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af794f0c-ba67-44ed-bff2-90448283e232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:26.204485 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af794f0c-ba67-44ed-bff2-90448283e232 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:26.204765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af794f0c-ba67-44ed-bff2-90448283e232 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:26.205064 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af794f0c-ba67-44ed-bff2-90448283e232 admin admin] PATH_INFO: `/v3/projects/63d54cc1acf24717968bbca9125265fb/users/fae8f3c24bff4457b3320be77d34e2f4/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:26.208043 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:26.222349 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af794f0c-ba67-44ed-bff2-90448283e232 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63d54cc1acf24717968bbca9125265fb, user_id=fae8f3c24bff4457b3320be77d34e2f4, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:26.225046 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af794f0c-ba67-44ed-bff2-90448283e232 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:26.237839 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1427/2844] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 14:02:26 2022] PUT /identity/v3/projects/63d54cc1acf24717968bbca9125265fb/users/fae8f3c24bff4457b3320be77d34e2f4/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 14:02:26.248801 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d2a5f-8435-4ade-90f4-e41ae151c996 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:26.249137 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d2a5f-8435-4ade-90f4-e41ae151c996 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:26.249421 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d2a5f-8435-4ade-90f4-e41ae151c996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:26.275531 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-2d2d2a5f-8435-4ade-90f4-e41ae151c996 None None] MFA Rules not processed for user `fae8f3c24bff4457b3320be77d34e2f4`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:02:26.276711 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-2d2d2a5f-8435-4ade-90f4-e41ae151c996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:26.339572 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1418/2845] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 14:02:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 14:02:26.344738 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af794f0c-ba67-44ed-bff2-90448283e232 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:26.405705 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af794f0c-ba67-44ed-bff2-90448283e232 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:26.410146 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e37ba9-0d69-40c6-b62c-859a2f22a760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:26.410794 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e37ba9-0d69-40c6-b62c-859a2f22a760 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:26.411077 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e37ba9-0d69-40c6-b62c-859a2f22a760 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:26.411355 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e37ba9-0d69-40c6-b62c-859a2f22a760 admin admin] PATH_INFO: `/v3/projects` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:26.412050 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e37ba9-0d69-40c6-b62c-859a2f22a760 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:26.413951 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e37ba9-0d69-40c6-b62c-859a2f22a760 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:26.445268 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1428/2846] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 14:02:26 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 14:02:26.449436 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d2a5f-8435-4ade-90f4-e41ae151c996 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:26.460548 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d2a5f-8435-4ade-90f4-e41ae151c996 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:26.486279 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:26.516833 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e984c43-88e1-4e32-b6ac-5d7cfef7fcbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:26.517553 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e984c43-88e1-4e32-b6ac-5d7cfef7fcbe admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:26.517841 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e984c43-88e1-4e32-b6ac-5d7cfef7fcbe admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:26.518115 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e984c43-88e1-4e32-b6ac-5d7cfef7fcbe admin admin] PATH_INFO: `/v3/users` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:26.518799 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e984c43-88e1-4e32-b6ac-5d7cfef7fcbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:26.520696 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e984c43-88e1-4e32-b6ac-5d7cfef7fcbe admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:26.539738 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1419/2847] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 14:02:26 2022] POST /identity/v3/users => generated 300 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 14:02:26.543776 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e37ba9-0d69-40c6-b62c-859a2f22a760 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:26.554495 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e37ba9-0d69-40c6-b62c-859a2f22a760 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:26.558989 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084523d-4796-43ed-99b3-f1e9d5efd20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:26.559637 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084523d-4796-43ed-99b3-f1e9d5efd20f admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:26.559918 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084523d-4796-43ed-99b3-f1e9d5efd20f admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:26.560866 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084523d-4796-43ed-99b3-f1e9d5efd20f admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:26.561556 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6084523d-4796-43ed-99b3-f1e9d5efd20f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:26.563126 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6084523d-4796-43ed-99b3-f1e9d5efd20f admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:26.573675 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1429/2848] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 14:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 14:02:26.577955 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e984c43-88e1-4e32-b6ac-5d7cfef7fcbe admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:26.587803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e984c43-88e1-4e32-b6ac-5d7cfef7fcbe admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:26.592541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdbaf8-114e-452f-a3a6-5a41c229d532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:26.593448 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdbaf8-114e-452f-a3a6-5a41c229d532 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:26.593735 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdbaf8-114e-452f-a3a6-5a41c229d532 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:26.594013 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdbaf8-114e-452f-a3a6-5a41c229d532 admin admin] PATH_INFO: `/v3/projects/6794fc2644bb4a6fa65f1919dbfac581/users/56d31c02f2a24db7bdb4f7b46d71ebf6/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:26.609145 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdbaf8-114e-452f-a3a6-5a41c229d532 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6794fc2644bb4a6fa65f1919dbfac581, user_id=56d31c02f2a24db7bdb4f7b46d71ebf6, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:26.611803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdbaf8-114e-452f-a3a6-5a41c229d532 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:26.624936 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1420/2849] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 14:02:26 2022] PUT /identity/v3/projects/6794fc2644bb4a6fa65f1919dbfac581/users/56d31c02f2a24db7bdb4f7b46d71ebf6/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:26.628948 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084523d-4796-43ed-99b3-f1e9d5efd20f admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:26.639410 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084523d-4796-43ed-99b3-f1e9d5efd20f admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:26.698439 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba33dbc-8faf-49e7-bc38-08d661bf2769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:26.699018 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba33dbc-8faf-49e7-bc38-08d661bf2769 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:26.699299 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba33dbc-8faf-49e7-bc38-08d661bf2769 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:26.699573 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba33dbc-8faf-49e7-bc38-08d661bf2769 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:26.700285 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba33dbc-8faf-49e7-bc38-08d661bf2769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:26.701952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba33dbc-8faf-49e7-bc38-08d661bf2769 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:26.711441 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1430/2850] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 14:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 14:02:26.715664 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdbaf8-114e-452f-a3a6-5a41c229d532 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:26.726727 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdbaf8-114e-452f-a3a6-5a41c229d532 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:26.731324 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ddbae-b7d6-41fb-8c63-0cd62b3d8c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:26.732783 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ddbae-b7d6-41fb-8c63-0cd62b3d8c80 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:26.733094 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ddbae-b7d6-41fb-8c63-0cd62b3d8c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:26.733373 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ddbae-b7d6-41fb-8c63-0cd62b3d8c80 admin admin] PATH_INFO: `/v3/projects/6794fc2644bb4a6fa65f1919dbfac581/users/56d31c02f2a24db7bdb4f7b46d71ebf6/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:26.735321 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ddbae-b7d6-41fb-8c63-0cd62b3d8c80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6794fc2644bb4a6fa65f1919dbfac581, user_id=56d31c02f2a24db7bdb4f7b46d71ebf6, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:26.737980 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ddbae-b7d6-41fb-8c63-0cd62b3d8c80 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:26.748126 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1421/2851] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 14:02:26 2022] PUT /identity/v3/projects/6794fc2644bb4a6fa65f1919dbfac581/users/56d31c02f2a24db7bdb4f7b46d71ebf6/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:26.759015 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11d29c-6c3d-4e6b-be64-268caac22724 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:26.759514 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11d29c-6c3d-4e6b-be64-268caac22724 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:26.759643 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11d29c-6c3d-4e6b-be64-268caac22724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:26.785659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-2b11d29c-6c3d-4e6b-be64-268caac22724 None None] MFA Rules not processed for user `56d31c02f2a24db7bdb4f7b46d71ebf6`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:02:26.786853 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-2b11d29c-6c3d-4e6b-be64-268caac22724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:26.830106 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:26.856233 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1431/2852] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 14:02:26 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 14:02:26.867971 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ddbae-b7d6-41fb-8c63-0cd62b3d8c80 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:26.869814 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-fd2ddbae-b7d6-41fb-8c63-0cd62b3d8c80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:26.926484 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ddbae-b7d6-41fb-8c63-0cd62b3d8c80 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:26.930413 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972dee5-7476-48ae-ae8e-540e2b8253dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:26.931052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972dee5-7476-48ae-ae8e-540e2b8253dc service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:26.931339 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972dee5-7476-48ae-ae8e-540e2b8253dc service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:26.931619 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972dee5-7476-48ae-ae8e-540e2b8253dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:26.941154 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b972dee5-7476-48ae-ae8e-540e2b8253dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:26.942588 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b972dee5-7476-48ae-ae8e-540e2b8253dc service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:26.952107 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1422/2853] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 14:02:26 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 14:02:27.014434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11d29c-6c3d-4e6b-be64-268caac22724 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:27.076879 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11d29c-6c3d-4e6b-be64-268caac22724 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:27.081451 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b7fe34-42c7-4621-abc7-e51d987e429e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:27.082313 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7fe34-42c7-4621-abc7-e51d987e429e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:27.082594 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7fe34-42c7-4621-abc7-e51d987e429e admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:27.082867 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7fe34-42c7-4621-abc7-e51d987e429e admin admin] PATH_INFO: `/v3/users/fae8f3c24bff4457b3320be77d34e2f4` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:27.084061 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b7fe34-42c7-4621-abc7-e51d987e429e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fae8f3c24bff4457b3320be77d34e2f4)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:27.086018 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b7fe34-42c7-4621-abc7-e51d987e429e admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:27.132413 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-77b7fe34-42c7-4621-abc7-e51d987e429e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fae8f3c24bff4457b3320be77d34e2f4'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:27.139990 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-77b7fe34-42c7-4621-abc7-e51d987e429e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fae8f3c24bff4457b3320be77d34e2f4'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:27.146873 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-77b7fe34-42c7-4621-abc7-e51d987e429e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fae8f3c24bff4457b3320be77d34e2f4'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:27.147530 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-77b7fe34-42c7-4621-abc7-e51d987e429e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fae8f3c24bff4457b3320be77d34e2f4'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:27.159820 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-77b7fe34-42c7-4621-abc7-e51d987e429e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fae8f3c24bff4457b3320be77d34e2f4'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:27.165593 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1432/2854] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 14:02:27 2022] DELETE /identity/v3/users/fae8f3c24bff4457b3320be77d34e2f4 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:27.170321 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bae383-0420-48d9-9fd6-a11737aaa120 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:27.170813 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bae383-0420-48d9-9fd6-a11737aaa120 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:27.171308 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bae383-0420-48d9-9fd6-a11737aaa120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:27.192416 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-42bae383-0420-48d9-9fd6-a11737aaa120 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:02:27.193656 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-42bae383-0420-48d9-9fd6-a11737aaa120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:27.251571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1423/2855] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 14:02:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 14:02:27.261218 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b7fe34-42c7-4621-abc7-e51d987e429e admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:27.262611 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-77b7fe34-42c7-4621-abc7-e51d987e429e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:27.321881 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b7fe34-42c7-4621-abc7-e51d987e429e admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:27.325506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab878d2-ec5e-410a-94b8-abfbb9628683 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:27.326750 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab878d2-ec5e-410a-94b8-abfbb9628683 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:27.327062 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab878d2-ec5e-410a-94b8-abfbb9628683 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:27.327337 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab878d2-ec5e-410a-94b8-abfbb9628683 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:27.336123 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab878d2-ec5e-410a-94b8-abfbb9628683 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:27.337561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab878d2-ec5e-410a-94b8-abfbb9628683 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:27.347625 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1433/2856] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 14:02:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 14:02:27.610678 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bae383-0420-48d9-9fd6-a11737aaa120 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:27.612118 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-42bae383-0420-48d9-9fd6-a11737aaa120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:27.626031 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bae383-0420-48d9-9fd6-a11737aaa120 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:27.630661 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dcb1a3-9faa-4e74-a558-17fe602cebec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:27.631519 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcb1a3-9faa-4e74-a558-17fe602cebec admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:27.631803 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcb1a3-9faa-4e74-a558-17fe602cebec admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:27.632577 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcb1a3-9faa-4e74-a558-17fe602cebec admin admin] PATH_INFO: `/v3/projects/63d54cc1acf24717968bbca9125265fb` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:27.633985 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dcb1a3-9faa-4e74-a558-17fe602cebec admin admin] RBAC: Authorizing `identity:delete_project(project_id=63d54cc1acf24717968bbca9125265fb)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:27.635926 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dcb1a3-9faa-4e74-a558-17fe602cebec admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:27.658104 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:27.683854 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-83dcb1a3-9faa-4e74-a558-17fe602cebec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63d54cc1acf24717968bbca9125265fb', 'request_id': 'req-83dcb1a3-9faa-4e74-a558-17fe602cebec'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:27.690595 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-83dcb1a3-9faa-4e74-a558-17fe602cebec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63d54cc1acf24717968bbca9125265fb', 'request_id': 'req-83dcb1a3-9faa-4e74-a558-17fe602cebec'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:27.697560 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-83dcb1a3-9faa-4e74-a558-17fe602cebec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63d54cc1acf24717968bbca9125265fb', 'request_id': 'req-83dcb1a3-9faa-4e74-a558-17fe602cebec'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:27.708450 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-83dcb1a3-9faa-4e74-a558-17fe602cebec admin admin] The token cache is being invalidate because project 63d54cc1acf24717968bbca9125265fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 14:02:27.708740 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-83dcb1a3-9faa-4e74-a558-17fe602cebec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:27.709386 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-83dcb1a3-9faa-4e74-a558-17fe602cebec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:27.710910 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1424/2857] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 14:02:27 2022] DELETE /identity/v3/projects/63d54cc1acf24717968bbca9125265fb => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:27.714861 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab878d2-ec5e-410a-94b8-abfbb9628683 service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:27.719910 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:27.725787 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab878d2-ec5e-410a-94b8-abfbb9628683 service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:27.782230 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4859ac4-cf5e-4ffa-9942-e97ed6666576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:27.783099 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4859ac4-cf5e-4ffa-9942-e97ed6666576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:27.783430 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4859ac4-cf5e-4ffa-9942-e97ed6666576 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:27.783652 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4859ac4-cf5e-4ffa-9942-e97ed6666576 admin admin] PATH_INFO: `/v3/users/56d31c02f2a24db7bdb4f7b46d71ebf6` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:27.784808 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4859ac4-cf5e-4ffa-9942-e97ed6666576 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56d31c02f2a24db7bdb4f7b46d71ebf6)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:27.786747 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4859ac4-cf5e-4ffa-9942-e97ed6666576 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:27.832596 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-e4859ac4-cf5e-4ffa-9942-e97ed6666576 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56d31c02f2a24db7bdb4f7b46d71ebf6'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:27.839200 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-e4859ac4-cf5e-4ffa-9942-e97ed6666576 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56d31c02f2a24db7bdb4f7b46d71ebf6'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:27.846172 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-e4859ac4-cf5e-4ffa-9942-e97ed6666576 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56d31c02f2a24db7bdb4f7b46d71ebf6'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:27.846808 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-e4859ac4-cf5e-4ffa-9942-e97ed6666576 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56d31c02f2a24db7bdb4f7b46d71ebf6'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:27.859125 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-e4859ac4-cf5e-4ffa-9942-e97ed6666576 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56d31c02f2a24db7bdb4f7b46d71ebf6'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:27.864918 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1434/2858] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 14:02:27 2022] DELETE /identity/v3/users/56d31c02f2a24db7bdb4f7b46d71ebf6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:28.134402 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dcb1a3-9faa-4e74-a558-17fe602cebec admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:28.135931 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-83dcb1a3-9faa-4e74-a558-17fe602cebec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:28.148942 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dcb1a3-9faa-4e74-a558-17fe602cebec admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:28.206281 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d763d-0182-42f7-a18d-85d3262a4537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:28.207142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d763d-0182-42f7-a18d-85d3262a4537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:28.207425 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d763d-0182-42f7-a18d-85d3262a4537 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:28.207702 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d763d-0182-42f7-a18d-85d3262a4537 admin admin] PATH_INFO: `/v3/projects/6794fc2644bb4a6fa65f1919dbfac581` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:28.208867 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d763d-0182-42f7-a18d-85d3262a4537 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6794fc2644bb4a6fa65f1919dbfac581)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:28.210807 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d763d-0182-42f7-a18d-85d3262a4537 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:28.257314 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-407d763d-0182-42f7-a18d-85d3262a4537 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6794fc2644bb4a6fa65f1919dbfac581', 'request_id': 'req-407d763d-0182-42f7-a18d-85d3262a4537'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:28.263501 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-407d763d-0182-42f7-a18d-85d3262a4537 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6794fc2644bb4a6fa65f1919dbfac581', 'request_id': 'req-407d763d-0182-42f7-a18d-85d3262a4537'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:28.272553 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-407d763d-0182-42f7-a18d-85d3262a4537 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6794fc2644bb4a6fa65f1919dbfac581', 'request_id': 'req-407d763d-0182-42f7-a18d-85d3262a4537'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:28.283268 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-407d763d-0182-42f7-a18d-85d3262a4537 admin admin] The token cache is being invalidate because project 6794fc2644bb4a6fa65f1919dbfac581 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 14:02:28.283553 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-407d763d-0182-42f7-a18d-85d3262a4537 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:28.284166 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-407d763d-0182-42f7-a18d-85d3262a4537 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:28.285765 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1425/2859] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 14:02:28 2022] DELETE /identity/v3/projects/6794fc2644bb4a6fa65f1919dbfac581 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:28.290657 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08768b8c-a676-4cf2-9876-b8fb152f3837 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:28.290952 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08768b8c-a676-4cf2-9876-b8fb152f3837 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:28.291231 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08768b8c-a676-4cf2-9876-b8fb152f3837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:28.310671 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-08768b8c-a676-4cf2-9876-b8fb152f3837 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:02:28.311797 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-08768b8c-a676-4cf2-9876-b8fb152f3837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:28.370214 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1435/2860] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 14:02:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 14:02:28.377965 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26261b9-63a6-4035-ba03-d7c64c164291 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:28.378257 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26261b9-63a6-4035-ba03-d7c64c164291 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:28.378552 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26261b9-63a6-4035-ba03-d7c64c164291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:28.396349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-a26261b9-63a6-4035-ba03-d7c64c164291 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:02:28.397554 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-a26261b9-63a6-4035-ba03-d7c64c164291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:28.405853 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1426/2861] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 14:02:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 14:02:28.409894 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08768b8c-a676-4cf2-9876-b8fb152f3837 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:28.419440 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08768b8c-a676-4cf2-9876-b8fb152f3837 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:28.424784 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391856a4-78d5-4665-993f-e1a2e39677a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:28.425476 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391856a4-78d5-4665-993f-e1a2e39677a4 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:28.425768 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391856a4-78d5-4665-993f-e1a2e39677a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:28.426048 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391856a4-78d5-4665-993f-e1a2e39677a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:28.426774 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391856a4-78d5-4665-993f-e1a2e39677a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 14:02:28.427085 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391856a4-78d5-4665-993f-e1a2e39677a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:28.428912 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391856a4-78d5-4665-993f-e1a2e39677a4 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:28.440373 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1436/2862] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 14:02:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 14:02:28.444545 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26261b9-63a6-4035-ba03-d7c64c164291 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:28.453959 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26261b9-63a6-4035-ba03-d7c64c164291 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:28.458641 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35babae-17a6-43b8-b31d-c22b12b1b476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:28.459276 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35babae-17a6-43b8-b31d-c22b12b1b476 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:28.459560 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35babae-17a6-43b8-b31d-c22b12b1b476 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:28.459842 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35babae-17a6-43b8-b31d-c22b12b1b476 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:28.460543 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35babae-17a6-43b8-b31d-c22b12b1b476 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:28.462475 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35babae-17a6-43b8-b31d-c22b12b1b476 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:28.488701 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1427/2863] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 14:02:28 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 14:02:28.492680 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391856a4-78d5-4665-993f-e1a2e39677a4 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:28.502750 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391856a4-78d5-4665-993f-e1a2e39677a4 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:28.507224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:28.562659 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a822e-4af4-49e5-a5f6-126432e5315e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:28.563245 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a822e-4af4-49e5-a5f6-126432e5315e admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:28.563530 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a822e-4af4-49e5-a5f6-126432e5315e admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:28.563808 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a822e-4af4-49e5-a5f6-126432e5315e admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:28.564503 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a822e-4af4-49e5-a5f6-126432e5315e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:28.566421 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a822e-4af4-49e5-a5f6-126432e5315e admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:28.586605 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1437/2864] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 14:02:28 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 14:02:28.590665 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35babae-17a6-43b8-b31d-c22b12b1b476 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:28.601263 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35babae-17a6-43b8-b31d-c22b12b1b476 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:28.605764 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49369c0-2666-4f54-bd60-2bff3070caed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:28.606411 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49369c0-2666-4f54-bd60-2bff3070caed admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:28.606695 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49369c0-2666-4f54-bd60-2bff3070caed admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:28.606969 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49369c0-2666-4f54-bd60-2bff3070caed admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:28.607765 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49369c0-2666-4f54-bd60-2bff3070caed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:28.609388 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49369c0-2666-4f54-bd60-2bff3070caed admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:28.611319 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:28.620840 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1428/2865] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 14:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 14:02:28.624488 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a822e-4af4-49e5-a5f6-126432e5315e admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:28.634075 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a822e-4af4-49e5-a5f6-126432e5315e admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:28.639104 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb368cf9-9a87-4a6e-bc70-5bc4afa4abe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:28.639747 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb368cf9-9a87-4a6e-bc70-5bc4afa4abe3 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:28.640027 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb368cf9-9a87-4a6e-bc70-5bc4afa4abe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:28.640297 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb368cf9-9a87-4a6e-bc70-5bc4afa4abe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:28.641035 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb368cf9-9a87-4a6e-bc70-5bc4afa4abe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:28.642707 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb368cf9-9a87-4a6e-bc70-5bc4afa4abe3 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:28.652460 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1438/2866] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 14:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 14:02:28.657217 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49369c0-2666-4f54-bd60-2bff3070caed admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:28.667175 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49369c0-2666-4f54-bd60-2bff3070caed admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:28.671773 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8afff1-07be-4313-8072-9e35c157aca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:28.672639 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8afff1-07be-4313-8072-9e35c157aca9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:28.672921 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8afff1-07be-4313-8072-9e35c157aca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:28.673231 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8afff1-07be-4313-8072-9e35c157aca9 admin admin] PATH_INFO: `/v3/projects/55ca81370c7f4b089954411c65f7a83a/users/d9aff8aa98c0467a9e7b57e9c7f23a14/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:28.687907 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8afff1-07be-4313-8072-9e35c157aca9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55ca81370c7f4b089954411c65f7a83a, user_id=d9aff8aa98c0467a9e7b57e9c7f23a14, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:28.690681 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8afff1-07be-4313-8072-9e35c157aca9 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:28.704331 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1429/2867] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 14:02:28 2022] PUT /identity/v3/projects/55ca81370c7f4b089954411c65f7a83a/users/d9aff8aa98c0467a9e7b57e9c7f23a14/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:28.714561 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013de64-5c15-4c35-afd3-01d19c270682 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:28.714850 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013de64-5c15-4c35-afd3-01d19c270682 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:28.715130 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013de64-5c15-4c35-afd3-01d19c270682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:28.741386 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-a013de64-5c15-4c35-afd3-01d19c270682 None None] MFA Rules not processed for user `d9aff8aa98c0467a9e7b57e9c7f23a14`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:02:28.742596 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-a013de64-5c15-4c35-afd3-01d19c270682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:28.804949 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1439/2868] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 14:02:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 14:02:28.809419 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8afff1-07be-4313-8072-9e35c157aca9 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:28.870398 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8afff1-07be-4313-8072-9e35c157aca9 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:28.875110 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bfb8a-f58d-438c-934e-96d7337993e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:28.875813 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bfb8a-f58d-438c-934e-96d7337993e7 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:28.876097 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bfb8a-f58d-438c-934e-96d7337993e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:28.876372 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bfb8a-f58d-438c-934e-96d7337993e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:28.877092 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3bfb8a-f58d-438c-934e-96d7337993e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:28.879002 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3bfb8a-f58d-438c-934e-96d7337993e7 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:28.903699 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1430/2869] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 14:02:28 2022] POST /identity/v3/projects => generated 379 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 14:02:28.908281 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013de64-5c15-4c35-afd3-01d19c270682 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:28.918709 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013de64-5c15-4c35-afd3-01d19c270682 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:28.976249 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d41afe-2485-4679-bc06-0b3282f708fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:28.976920 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d41afe-2485-4679-bc06-0b3282f708fe admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:28.977237 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d41afe-2485-4679-bc06-0b3282f708fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:28.977516 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d41afe-2485-4679-bc06-0b3282f708fe admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:28.978218 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d41afe-2485-4679-bc06-0b3282f708fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:28.980144 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d41afe-2485-4679-bc06-0b3282f708fe admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:29.001679 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1440/2870] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 14:02:28 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 14:02:29.005576 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bfb8a-f58d-438c-934e-96d7337993e7 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:29.016386 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bfb8a-f58d-438c-934e-96d7337993e7 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:29.020965 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef73ac5-775e-471c-bc85-21cbe74f3ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:29.021654 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef73ac5-775e-471c-bc85-21cbe74f3ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:29.021943 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef73ac5-775e-471c-bc85-21cbe74f3ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:29.022225 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef73ac5-775e-471c-bc85-21cbe74f3ac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:29.022975 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef73ac5-775e-471c-bc85-21cbe74f3ac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:29.024568 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef73ac5-775e-471c-bc85-21cbe74f3ac6 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:29.035626 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1431/2871] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 14:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 14:02:29.039510 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d41afe-2485-4679-bc06-0b3282f708fe admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:29.049409 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d41afe-2485-4679-bc06-0b3282f708fe admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:29.054976 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568f6c1-33c7-4b89-a545-a2e121f302a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:29.055636 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568f6c1-33c7-4b89-a545-a2e121f302a5 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:29.055919 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568f6c1-33c7-4b89-a545-a2e121f302a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:29.056202 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568f6c1-33c7-4b89-a545-a2e121f302a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:29.056938 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e568f6c1-33c7-4b89-a545-a2e121f302a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:29.058551 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e568f6c1-33c7-4b89-a545-a2e121f302a5 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:29.059843 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:29.070629 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1441/2872] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 14:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 14:02:29.074653 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef73ac5-775e-471c-bc85-21cbe74f3ac6 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:29.085176 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef73ac5-775e-471c-bc85-21cbe74f3ac6 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:29.089701 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426e8dd-c2f4-418d-8db1-01bf365e2bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:29.090571 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426e8dd-c2f4-418d-8db1-01bf365e2bae admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:29.090854 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426e8dd-c2f4-418d-8db1-01bf365e2bae admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:29.091147 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426e8dd-c2f4-418d-8db1-01bf365e2bae admin admin] PATH_INFO: `/v3/projects/4ef667cc3e75438494561f33bcf4ebd6/users/c1314ea24c3d47c7ac74b2695de2be6e/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:29.105240 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3426e8dd-c2f4-418d-8db1-01bf365e2bae admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ef667cc3e75438494561f33bcf4ebd6, user_id=c1314ea24c3d47c7ac74b2695de2be6e, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:29.108448 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3426e8dd-c2f4-418d-8db1-01bf365e2bae admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:29.121383 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1432/2873] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 14:02:29 2022] PUT /identity/v3/projects/4ef667cc3e75438494561f33bcf4ebd6/users/c1314ea24c3d47c7ac74b2695de2be6e/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:29.132839 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6412c1-bb50-4574-aeb1-f29c6f6c7c5b None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:29.133173 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6412c1-bb50-4574-aeb1-f29c6f6c7c5b None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:29.133451 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6412c1-bb50-4574-aeb1-f29c6f6c7c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:29.159935 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-6b6412c1-bb50-4574-aeb1-f29c6f6c7c5b None None] MFA Rules not processed for user `c1314ea24c3d47c7ac74b2695de2be6e`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:02:29.161158 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-6b6412c1-bb50-4574-aeb1-f29c6f6c7c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:29.224973 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1442/2874] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 14:02:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 14:02:29.229398 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426e8dd-c2f4-418d-8db1-01bf365e2bae admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:29.289579 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426e8dd-c2f4-418d-8db1-01bf365e2bae admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:29.294214 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6d0ad-a258-40f6-a142-411298845e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:29.295918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6d0ad-a258-40f6-a142-411298845e95 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:29.296304 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6d0ad-a258-40f6-a142-411298845e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:29.296304 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6d0ad-a258-40f6-a142-411298845e95 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:29.297248 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab6d0ad-a258-40f6-a142-411298845e95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:29.298860 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab6d0ad-a258-40f6-a142-411298845e95 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:29.324277 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1433/2875] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 14:02:29 2022] POST /identity/v3/projects => generated 379 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 14:02:29.328752 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6412c1-bb50-4574-aeb1-f29c6f6c7c5b None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:29.339404 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6412c1-bb50-4574-aeb1-f29c6f6c7c5b None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:29.394773 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b33bb4b-90c1-4a34-8853-d9e1ff1c077e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:29.395368 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33bb4b-90c1-4a34-8853-d9e1ff1c077e admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:29.395652 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33bb4b-90c1-4a34-8853-d9e1ff1c077e admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:29.395929 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33bb4b-90c1-4a34-8853-d9e1ff1c077e admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:29.396630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b33bb4b-90c1-4a34-8853-d9e1ff1c077e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:29.398576 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b33bb4b-90c1-4a34-8853-d9e1ff1c077e admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:29.418596 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1443/2876] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 14:02:29 2022] POST /identity/v3/users => generated 296 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 14:02:29.422520 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6d0ad-a258-40f6-a142-411298845e95 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:29.432985 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6d0ad-a258-40f6-a142-411298845e95 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:29.438349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b272b1-4768-4e40-8930-58949bfaa7e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:29.439005 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b272b1-4768-4e40-8930-58949bfaa7e0 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:29.439291 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b272b1-4768-4e40-8930-58949bfaa7e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:29.439599 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b272b1-4768-4e40-8930-58949bfaa7e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:29.440326 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b272b1-4768-4e40-8930-58949bfaa7e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:29.441949 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b272b1-4768-4e40-8930-58949bfaa7e0 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:29.444014 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:29.456019 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1434/2877] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 14:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 14:02:29.460005 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b33bb4b-90c1-4a34-8853-d9e1ff1c077e admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:29.470320 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b33bb4b-90c1-4a34-8853-d9e1ff1c077e admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:29.474858 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ecd65-c632-4682-9be0-45f6f5ceb415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:29.475730 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ecd65-c632-4682-9be0-45f6f5ceb415 admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:29.476012 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ecd65-c632-4682-9be0-45f6f5ceb415 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:29.476293 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ecd65-c632-4682-9be0-45f6f5ceb415 admin admin] PATH_INFO: `/v3/projects/f0cad106340e458599c4c7150568760e/users/fd23ce57f77b428a9a77c0b4f819f9ea/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:29.490616 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ecd65-c632-4682-9be0-45f6f5ceb415 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0cad106340e458599c4c7150568760e, user_id=fd23ce57f77b428a9a77c0b4f819f9ea, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:29.493271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ecd65-c632-4682-9be0-45f6f5ceb415 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:29.506294 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1444/2878] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 14:02:29 2022] PUT /identity/v3/projects/f0cad106340e458599c4c7150568760e/users/fd23ce57f77b428a9a77c0b4f819f9ea/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:29.510052 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b272b1-4768-4e40-8930-58949bfaa7e0 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:29.520668 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b272b1-4768-4e40-8930-58949bfaa7e0 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:29.576423 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53cce2-7ae2-4437-92d3-07f8cda8b81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:29.577111 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53cce2-7ae2-4437-92d3-07f8cda8b81c admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:29.577394 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53cce2-7ae2-4437-92d3-07f8cda8b81c admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:29.577668 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53cce2-7ae2-4437-92d3-07f8cda8b81c admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:29.578380 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c53cce2-7ae2-4437-92d3-07f8cda8b81c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:29.579982 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c53cce2-7ae2-4437-92d3-07f8cda8b81c admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:29.589724 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1435/2879] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 14:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 14:02:29.595008 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ecd65-c632-4682-9be0-45f6f5ceb415 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:29.605281 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ecd65-c632-4682-9be0-45f6f5ceb415 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:29.610306 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a75a9-f4fc-4924-89ec-6ae7741d21ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:29.611184 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a75a9-f4fc-4924-89ec-6ae7741d21ef admin admin] REQUEST_METHOD: `PUT` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:29.611472 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a75a9-f4fc-4924-89ec-6ae7741d21ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:29.611752 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a75a9-f4fc-4924-89ec-6ae7741d21ef admin admin] PATH_INFO: `/v3/projects/f0cad106340e458599c4c7150568760e/users/fd23ce57f77b428a9a77c0b4f819f9ea/roles/795daa2c78354fd48e4a741e1182cc98` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:29.613742 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259a75a9-f4fc-4924-89ec-6ae7741d21ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0cad106340e458599c4c7150568760e, user_id=fd23ce57f77b428a9a77c0b4f819f9ea, role_id=795daa2c78354fd48e4a741e1182cc98)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:29.616366 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259a75a9-f4fc-4924-89ec-6ae7741d21ef admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:29.626807 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1445/2880] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 14:02:29 2022] PUT /identity/v3/projects/f0cad106340e458599c4c7150568760e/users/fd23ce57f77b428a9a77c0b4f819f9ea/roles/795daa2c78354fd48e4a741e1182cc98 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:29.636635 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba416d5d-dd23-451b-8adc-baff32662e05 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:29.636933 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba416d5d-dd23-451b-8adc-baff32662e05 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:29.637251 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba416d5d-dd23-451b-8adc-baff32662e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:29.665424 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-ba416d5d-dd23-451b-8adc-baff32662e05 None None] MFA Rules not processed for user `fd23ce57f77b428a9a77c0b4f819f9ea`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:02:29.666553 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-ba416d5d-dd23-451b-8adc-baff32662e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:29.738977 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1436/2881] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 14:02:29 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 14:02:29.750819 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a75a9-f4fc-4924-89ec-6ae7741d21ef admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:29.752553 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-259a75a9-f4fc-4924-89ec-6ae7741d21ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:29.796271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:29.813224 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a75a9-f4fc-4924-89ec-6ae7741d21ef admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:29.816833 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6178554b-cb72-4929-9326-9d7ca649bc7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:29.817510 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178554b-cb72-4929-9326-9d7ca649bc7d service cinder] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:29.817793 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178554b-cb72-4929-9326-9d7ca649bc7d service cinder] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:29.818068 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178554b-cb72-4929-9326-9d7ca649bc7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:29.827790 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6178554b-cb72-4929-9326-9d7ca649bc7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:29.829230 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6178554b-cb72-4929-9326-9d7ca649bc7d service cinder] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:29.838753 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1446/2882] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 14:02:29 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 14:02:29.878830 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba416d5d-dd23-451b-8adc-baff32662e05 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:29.880293 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-ba416d5d-dd23-451b-8adc-baff32662e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:29.892737 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba416d5d-dd23-451b-8adc-baff32662e05 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:29.896340 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf4cf1-c55b-4739-82e2-0ece9add1794 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:29.896979 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf4cf1-c55b-4739-82e2-0ece9add1794 service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:29.897295 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf4cf1-c55b-4739-82e2-0ece9add1794 service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:29.897574 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf4cf1-c55b-4739-82e2-0ece9add1794 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:29.955067 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf4cf1-c55b-4739-82e2-0ece9add1794 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:29.956464 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf4cf1-c55b-4739-82e2-0ece9add1794 service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:29.965457 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1437/2883] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 14:02:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 14:02:32.809411 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6178554b-cb72-4929-9326-9d7ca649bc7d service cinder] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:32.870535 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6178554b-cb72-4929-9326-9d7ca649bc7d service cinder] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:32.875491 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8353e811-ce47-4bdb-99e8-d5f0cbbea8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:32.876343 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353e811-ce47-4bdb-99e8-d5f0cbbea8dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:32.876620 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353e811-ce47-4bdb-99e8-d5f0cbbea8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:32.876888 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353e811-ce47-4bdb-99e8-d5f0cbbea8dc admin admin] PATH_INFO: `/v3/users/d9aff8aa98c0467a9e7b57e9c7f23a14` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:32.878077 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8353e811-ce47-4bdb-99e8-d5f0cbbea8dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9aff8aa98c0467a9e7b57e9c7f23a14)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:32.879957 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8353e811-ce47-4bdb-99e8-d5f0cbbea8dc admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:32.923720 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8353e811-ce47-4bdb-99e8-d5f0cbbea8dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9aff8aa98c0467a9e7b57e9c7f23a14'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:32.931286 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8353e811-ce47-4bdb-99e8-d5f0cbbea8dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9aff8aa98c0467a9e7b57e9c7f23a14'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:32.938779 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8353e811-ce47-4bdb-99e8-d5f0cbbea8dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9aff8aa98c0467a9e7b57e9c7f23a14'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:32.939421 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8353e811-ce47-4bdb-99e8-d5f0cbbea8dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9aff8aa98c0467a9e7b57e9c7f23a14'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:32.951643 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-8353e811-ce47-4bdb-99e8-d5f0cbbea8dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9aff8aa98c0467a9e7b57e9c7f23a14'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:32.957502 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1447/2884] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 14:02:32 2022] DELETE /identity/v3/users/d9aff8aa98c0467a9e7b57e9c7f23a14 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:32.962040 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83068511-020e-4d2f-8469-49959088e926 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:32.962534 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83068511-020e-4d2f-8469-49959088e926 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:32.962667 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83068511-020e-4d2f-8469-49959088e926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:32.979580 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-83068511-020e-4d2f-8469-49959088e926 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:02:32.980719 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-83068511-020e-4d2f-8469-49959088e926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:32.992835 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=100 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:33.040416 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1438/2885] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 14:02:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 14:02:33.051039 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8353e811-ce47-4bdb-99e8-d5f0cbbea8dc admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:33.053105 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-8353e811-ce47-4bdb-99e8-d5f0cbbea8dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:33.112764 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8353e811-ce47-4bdb-99e8-d5f0cbbea8dc admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:33.116333 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728dcdda-7e85-4af6-b632-f99944bf12c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:33.116980 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728dcdda-7e85-4af6-b632-f99944bf12c0 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:33.117296 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728dcdda-7e85-4af6-b632-f99944bf12c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:33.117572 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728dcdda-7e85-4af6-b632-f99944bf12c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:33.127489 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728dcdda-7e85-4af6-b632-f99944bf12c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:33.128961 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728dcdda-7e85-4af6-b632-f99944bf12c0 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:33.138434 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1448/2886] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 14:02:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 14:02:33.421657 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83068511-020e-4d2f-8469-49959088e926 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:33.423111 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-83068511-020e-4d2f-8469-49959088e926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:33.435838 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83068511-020e-4d2f-8469-49959088e926 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:33.441123 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a68bba-191a-4441-b598-5f1c5cad55dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:33.442194 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a68bba-191a-4441-b598-5f1c5cad55dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:33.442586 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a68bba-191a-4441-b598-5f1c5cad55dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:33.442936 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a68bba-191a-4441-b598-5f1c5cad55dd admin admin] PATH_INFO: `/v3/projects/55ca81370c7f4b089954411c65f7a83a` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:33.444255 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a68bba-191a-4441-b598-5f1c5cad55dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=55ca81370c7f4b089954411c65f7a83a)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:33.446361 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a68bba-191a-4441-b598-5f1c5cad55dd admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:33.493991 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-28a68bba-191a-4441-b598-5f1c5cad55dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55ca81370c7f4b089954411c65f7a83a', 'request_id': 'req-28a68bba-191a-4441-b598-5f1c5cad55dd'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:33.500622 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-28a68bba-191a-4441-b598-5f1c5cad55dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55ca81370c7f4b089954411c65f7a83a', 'request_id': 'req-28a68bba-191a-4441-b598-5f1c5cad55dd'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:33.507879 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-28a68bba-191a-4441-b598-5f1c5cad55dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55ca81370c7f4b089954411c65f7a83a', 'request_id': 'req-28a68bba-191a-4441-b598-5f1c5cad55dd'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:33.518839 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-28a68bba-191a-4441-b598-5f1c5cad55dd admin admin] The token cache is being invalidate because project 55ca81370c7f4b089954411c65f7a83a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 14:02:33.519136 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-28a68bba-191a-4441-b598-5f1c5cad55dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:33.519772 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-28a68bba-191a-4441-b598-5f1c5cad55dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:33.521358 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1439/2887] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 14:02:33 2022] DELETE /identity/v3/projects/55ca81370c7f4b089954411c65f7a83a => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:33.525271 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728dcdda-7e85-4af6-b632-f99944bf12c0 service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:33.530603 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:33.537193 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728dcdda-7e85-4af6-b632-f99944bf12c0 service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:33.593903 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685cfd6b-1957-4d70-ad6d-39d168917ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:33.594765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cfd6b-1957-4d70-ad6d-39d168917ec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:33.595046 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cfd6b-1957-4d70-ad6d-39d168917ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:33.595324 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cfd6b-1957-4d70-ad6d-39d168917ec5 admin admin] PATH_INFO: `/v3/users/c1314ea24c3d47c7ac74b2695de2be6e` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:33.596487 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685cfd6b-1957-4d70-ad6d-39d168917ec5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1314ea24c3d47c7ac74b2695de2be6e)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:33.598437 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685cfd6b-1957-4d70-ad6d-39d168917ec5 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:33.643301 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-685cfd6b-1957-4d70-ad6d-39d168917ec5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1314ea24c3d47c7ac74b2695de2be6e'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:33.650386 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-685cfd6b-1957-4d70-ad6d-39d168917ec5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1314ea24c3d47c7ac74b2695de2be6e'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:33.657744 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-685cfd6b-1957-4d70-ad6d-39d168917ec5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1314ea24c3d47c7ac74b2695de2be6e'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:33.658410 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-685cfd6b-1957-4d70-ad6d-39d168917ec5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1314ea24c3d47c7ac74b2695de2be6e'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:33.670765 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-685cfd6b-1957-4d70-ad6d-39d168917ec5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1314ea24c3d47c7ac74b2695de2be6e'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:33.676455 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1449/2888] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 14:02:33 2022] DELETE /identity/v3/users/c1314ea24c3d47c7ac74b2695de2be6e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:33.960322 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a68bba-191a-4441-b598-5f1c5cad55dd admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:33.961804 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-28a68bba-191a-4441-b598-5f1c5cad55dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:33.975523 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a68bba-191a-4441-b598-5f1c5cad55dd admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:34.036534 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ebf5b6-bcac-4ebd-aeda-f327ec757bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:34.037356 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ebf5b6-bcac-4ebd-aeda-f327ec757bcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:34.037642 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ebf5b6-bcac-4ebd-aeda-f327ec757bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:34.037918 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ebf5b6-bcac-4ebd-aeda-f327ec757bcb admin admin] PATH_INFO: `/v3/projects/4ef667cc3e75438494561f33bcf4ebd6` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:34.039095 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ebf5b6-bcac-4ebd-aeda-f327ec757bcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ef667cc3e75438494561f33bcf4ebd6)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:34.040985 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ebf5b6-bcac-4ebd-aeda-f327ec757bcb admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:34.088018 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:34.094105 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-09ebf5b6-bcac-4ebd-aeda-f327ec757bcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ef667cc3e75438494561f33bcf4ebd6', 'request_id': 'req-09ebf5b6-bcac-4ebd-aeda-f327ec757bcb'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:34.101875 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-09ebf5b6-bcac-4ebd-aeda-f327ec757bcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ef667cc3e75438494561f33bcf4ebd6', 'request_id': 'req-09ebf5b6-bcac-4ebd-aeda-f327ec757bcb'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:34.110233 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-09ebf5b6-bcac-4ebd-aeda-f327ec757bcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ef667cc3e75438494561f33bcf4ebd6', 'request_id': 'req-09ebf5b6-bcac-4ebd-aeda-f327ec757bcb'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:34.121142 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-09ebf5b6-bcac-4ebd-aeda-f327ec757bcb admin admin] The token cache is being invalidate because project 4ef667cc3e75438494561f33bcf4ebd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 14:02:34.121451 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-09ebf5b6-bcac-4ebd-aeda-f327ec757bcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:34.122064 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-09ebf5b6-bcac-4ebd-aeda-f327ec757bcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:34.124086 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1440/2889] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 14:02:33 2022] DELETE /identity/v3/projects/4ef667cc3e75438494561f33bcf4ebd6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 14:02:34.128070 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685cfd6b-1957-4d70-ad6d-39d168917ec5 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:34.129554 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-685cfd6b-1957-4d70-ad6d-39d168917ec5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:34.143420 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685cfd6b-1957-4d70-ad6d-39d168917ec5 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:34.199618 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee79c8-6fde-453b-a839-fac96ed03f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:34.201128 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee79c8-6fde-453b-a839-fac96ed03f4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:34.201462 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee79c8-6fde-453b-a839-fac96ed03f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:34.201744 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee79c8-6fde-453b-a839-fac96ed03f4e admin admin] PATH_INFO: `/v3/users/fd23ce57f77b428a9a77c0b4f819f9ea` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:34.202937 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee79c8-6fde-453b-a839-fac96ed03f4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd23ce57f77b428a9a77c0b4f819f9ea)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:34.204837 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee79c8-6fde-453b-a839-fac96ed03f4e admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:34.227318 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:34.254230 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-7cee79c8-6fde-453b-a839-fac96ed03f4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd23ce57f77b428a9a77c0b4f819f9ea'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:34.261328 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-7cee79c8-6fde-453b-a839-fac96ed03f4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd23ce57f77b428a9a77c0b4f819f9ea'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:34.268890 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-7cee79c8-6fde-453b-a839-fac96ed03f4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd23ce57f77b428a9a77c0b4f819f9ea'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:34.269565 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-7cee79c8-6fde-453b-a839-fac96ed03f4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd23ce57f77b428a9a77c0b4f819f9ea'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:34.282225 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-7cee79c8-6fde-453b-a839-fac96ed03f4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd23ce57f77b428a9a77c0b4f819f9ea'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:34.288124 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1450/2890] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 14:02:34 2022] DELETE /identity/v3/users/fd23ce57f77b428a9a77c0b4f819f9ea => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:34.557046 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ebf5b6-bcac-4ebd-aeda-f327ec757bcb admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:34.558594 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-09ebf5b6-bcac-4ebd-aeda-f327ec757bcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:34.572887 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ebf5b6-bcac-4ebd-aeda-f327ec757bcb admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:34.629842 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e93520-467f-4232-bf54-5c06ace6a42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:34.630700 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e93520-467f-4232-bf54-5c06ace6a42e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:34.630982 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e93520-467f-4232-bf54-5c06ace6a42e admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:34.631253 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e93520-467f-4232-bf54-5c06ace6a42e admin admin] PATH_INFO: `/v3/projects/f0cad106340e458599c4c7150568760e` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:34.632419 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e93520-467f-4232-bf54-5c06ace6a42e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0cad106340e458599c4c7150568760e)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:34.634386 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e93520-467f-4232-bf54-5c06ace6a42e admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:34.678767 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a3e93520-467f-4232-bf54-5c06ace6a42e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0cad106340e458599c4c7150568760e', 'request_id': 'req-a3e93520-467f-4232-bf54-5c06ace6a42e'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:34.684718 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a3e93520-467f-4232-bf54-5c06ace6a42e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0cad106340e458599c4c7150568760e', 'request_id': 'req-a3e93520-467f-4232-bf54-5c06ace6a42e'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:34.692299 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a3e93520-467f-4232-bf54-5c06ace6a42e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0cad106340e458599c4c7150568760e', 'request_id': 'req-a3e93520-467f-4232-bf54-5c06ace6a42e'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:34.703187 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a3e93520-467f-4232-bf54-5c06ace6a42e admin admin] The token cache is being invalidate because project f0cad106340e458599c4c7150568760e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 14:02:34.703500 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a3e93520-467f-4232-bf54-5c06ace6a42e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:34.704120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a3e93520-467f-4232-bf54-5c06ace6a42e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:34.705731 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1441/2891] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 14:02:34 2022] DELETE /identity/v3/projects/f0cad106340e458599c4c7150568760e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:34.711656 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03327107-d82f-4566-acb3-8c62ff9a802e None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:34.711943 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03327107-d82f-4566-acb3-8c62ff9a802e None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:34.712218 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03327107-d82f-4566-acb3-8c62ff9a802e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:34.731267 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-03327107-d82f-4566-acb3-8c62ff9a802e None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:02:34.733272 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-03327107-d82f-4566-acb3-8c62ff9a802e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:34.792902 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1451/2892] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 14:02:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 14:02:34.802367 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94cb02-003b-4126-947c-402fbf87f613 None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:34.802592 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94cb02-003b-4126-947c-402fbf87f613 None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:34.802876 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94cb02-003b-4126-947c-402fbf87f613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:34.822881 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-6d94cb02-003b-4126-947c-402fbf87f613 None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:02:34.824100 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-6d94cb02-003b-4126-947c-402fbf87f613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:34.833313 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1442/2893] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 14:02:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 14:02:34.837463 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03327107-d82f-4566-acb3-8c62ff9a802e None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:34.847075 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03327107-d82f-4566-acb3-8c62ff9a802e None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:34.851548 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9926de-7cb7-4ed6-918c-ba758e86f2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:34.852225 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9926de-7cb7-4ed6-918c-ba758e86f2e9 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:34.852506 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9926de-7cb7-4ed6-918c-ba758e86f2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:34.852783 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9926de-7cb7-4ed6-918c-ba758e86f2e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:34.853539 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9926de-7cb7-4ed6-918c-ba758e86f2e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86849) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 14:02:34.853873 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9926de-7cb7-4ed6-918c-ba758e86f2e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:34.855442 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9926de-7cb7-4ed6-918c-ba758e86f2e9 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:34.867196 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1452/2894] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jul 6 14:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 14:02:34.871439 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94cb02-003b-4126-947c-402fbf87f613 None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:34.881913 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94cb02-003b-4126-947c-402fbf87f613 None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:34.886513 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd695d-6800-469e-ab1d-25a2dff76ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:34.887158 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd695d-6800-469e-ab1d-25a2dff76ff1 admin admin] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:34.887439 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd695d-6800-469e-ab1d-25a2dff76ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:34.887716 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd695d-6800-469e-ab1d-25a2dff76ff1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:34.888891 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdd695d-6800-469e-ab1d-25a2dff76ff1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:34.890819 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdd695d-6800-469e-ab1d-25a2dff76ff1 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:34.915975 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1443/2895] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jul 6 14:02:34 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 14:02:34.920831 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9926de-7cb7-4ed6-918c-ba758e86f2e9 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:34.930872 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9926de-7cb7-4ed6-918c-ba758e86f2e9 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:34.986341 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3812b-7be4-4c4b-bda0-393f721e3442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:34.987001 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3812b-7be4-4c4b-bda0-393f721e3442 admin admin] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:34.987283 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3812b-7be4-4c4b-bda0-393f721e3442 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:34.987557 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3812b-7be4-4c4b-bda0-393f721e3442 admin admin] PATH_INFO: `/v3/users` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:34.988248 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c3812b-7be4-4c4b-bda0-393f721e3442 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:34.990174 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c3812b-7be4-4c4b-bda0-393f721e3442 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:35.011315 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1453/2896] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jul 6 14:02:34 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 14:02:35.015137 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd695d-6800-469e-ab1d-25a2dff76ff1 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:35.025370 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd695d-6800-469e-ab1d-25a2dff76ff1 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:35.030569 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657afa4-254c-477a-85b9-9a9325683ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:35.031227 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657afa4-254c-477a-85b9-9a9325683ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:35.031511 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657afa4-254c-477a-85b9-9a9325683ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:35.031789 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657afa4-254c-477a-85b9-9a9325683ac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:35.032541 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a657afa4-254c-477a-85b9-9a9325683ac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:35.034158 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a657afa4-254c-477a-85b9-9a9325683ac3 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:35.045094 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1444/2897] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 14:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 14:02:35.048801 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3812b-7be4-4c4b-bda0-393f721e3442 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:35.059089 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3812b-7be4-4c4b-bda0-393f721e3442 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:35.063808 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddd97fd-e745-43a9-9bb0-b94d175120e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:35.064479 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd97fd-e745-43a9-9bb0-b94d175120e5 admin admin] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:35.064768 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd97fd-e745-43a9-9bb0-b94d175120e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:35.065080 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd97fd-e745-43a9-9bb0-b94d175120e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:35.065816 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddd97fd-e745-43a9-9bb0-b94d175120e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:35.067456 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddd97fd-e745-43a9-9bb0-b94d175120e5 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:35.078630 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1454/2898] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jul 6 14:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 14:02:35.082797 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657afa4-254c-477a-85b9-9a9325683ac3 admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:35.095038 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657afa4-254c-477a-85b9-9a9325683ac3 admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:35.100625 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795fbb3-f30a-4b96-a0eb-51fb199f562d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:35.101546 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795fbb3-f30a-4b96-a0eb-51fb199f562d admin admin] REQUEST_METHOD: `PUT` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:35.101832 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795fbb3-f30a-4b96-a0eb-51fb199f562d admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:35.103117 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795fbb3-f30a-4b96-a0eb-51fb199f562d admin admin] PATH_INFO: `/v3/projects/a0d86d7a87e242b194b423c91adfff7c/users/68c68b54240444ffb90c5a33ece5bf76/roles/b3244d1d9f004c1b8d3594fd30da1024` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:35.119958 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795fbb3-f30a-4b96-a0eb-51fb199f562d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0d86d7a87e242b194b423c91adfff7c, user_id=68c68b54240444ffb90c5a33ece5bf76, role_id=b3244d1d9f004c1b8d3594fd30da1024)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:35.123204 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795fbb3-f30a-4b96-a0eb-51fb199f562d admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:35.130943 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:35.136600 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1445/2899] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jul 6 14:02:35 2022] PUT /identity/v3/projects/a0d86d7a87e242b194b423c91adfff7c/users/68c68b54240444ffb90c5a33ece5bf76/roles/b3244d1d9f004c1b8d3594fd30da1024 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:35.149858 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f41799-c845-4e69-abe7-e54585fe86d4 None None] REQUEST_METHOD: `POST` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:35.170435 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f41799-c845-4e69-abe7-e54585fe86d4 None None] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:35.170435 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f41799-c845-4e69-abe7-e54585fe86d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:35.170435 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:35.185542 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.auth.core [None req-54f41799-c845-4e69-abe7-e54585fe86d4 None None] MFA Rules not processed for user `68c68b54240444ffb90c5a33ece5bf76`. Rule list: `[]` (Enabled: `True`). {{(pid=86849) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:02:35.186985 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-54f41799-c845-4e69-abe7-e54585fe86d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:35.250781 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1455/2900] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 14:02:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 14:02:35.261902 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795fbb3-f30a-4b96-a0eb-51fb199f562d admin admin] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:35.264745 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-1795fbb3-f30a-4b96-a0eb-51fb199f562d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:35.321827 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795fbb3-f30a-4b96-a0eb-51fb199f562d admin admin] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:35.325349 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbed2d4-e01f-4f69-a9f1-8243a819bdc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5997058ea0fe456a89856e222722a5cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:35.326763 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbed2d4-e01f-4f69-a9f1-8243a819bdc2 service cinder] REQUEST_METHOD: `GET` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:35.327044 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbed2d4-e01f-4f69-a9f1-8243a819bdc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:35.327324 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbed2d4-e01f-4f69-a9f1-8243a819bdc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:35.335598 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbed2d4-e01f-4f69-a9f1-8243a819bdc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:35.337053 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbed2d4-e01f-4f69-a9f1-8243a819bdc2 service cinder] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:35.345973 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1446/2901] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 14:02:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 14:02:35.366593 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f41799-c845-4e69-abe7-e54585fe86d4 None None] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:35.430070 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f41799-c845-4e69-abe7-e54585fe86d4 None None] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:35.435488 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d15722b-93fa-40e4-bb96-7f34c178f570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:35.436383 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15722b-93fa-40e4-bb96-7f34c178f570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:35.436663 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15722b-93fa-40e4-bb96-7f34c178f570 admin admin] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:35.436942 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15722b-93fa-40e4-bb96-7f34c178f570 admin admin] PATH_INFO: `/v3/users/68c68b54240444ffb90c5a33ece5bf76` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:35.438185 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d15722b-93fa-40e4-bb96-7f34c178f570 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68c68b54240444ffb90c5a33ece5bf76)` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:35.440112 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d15722b-93fa-40e4-bb96-7f34c178f570 admin admin] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:35.485195 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-6d15722b-93fa-40e4-bb96-7f34c178f570 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68c68b54240444ffb90c5a33ece5bf76'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:35.492117 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-6d15722b-93fa-40e4-bb96-7f34c178f570 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68c68b54240444ffb90c5a33ece5bf76'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:35.499596 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-6d15722b-93fa-40e4-bb96-7f34c178f570 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68c68b54240444ffb90c5a33ece5bf76'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:35.500240 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-6d15722b-93fa-40e4-bb96-7f34c178f570 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68c68b54240444ffb90c5a33ece5bf76'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:35.511822 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.notifications [None req-6d15722b-93fa-40e4-bb96-7f34c178f570 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68c68b54240444ffb90c5a33ece5bf76'} {{(pid=86849) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:35.517889 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1456/2902] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jul 6 14:02:35 2022] DELETE /identity/v3/users/68c68b54240444ffb90c5a33ece5bf76 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:35.522348 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7870c-9de2-43a1-b9c5-b344fafb545e None None] REQUEST_METHOD: `POST` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:35.522642 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7870c-9de2-43a1-b9c5-b344fafb545e None None] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:35.522924 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7870c-9de2-43a1-b9c5-b344fafb545e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:35.539840 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.auth.core [None req-7ad7870c-9de2-43a1-b9c5-b344fafb545e None None] MFA Rules not processed for user `c17e13a3d58f4ceab768f46d43e2cab5`. Rule list: `[]` (Enabled: `True`). {{(pid=86850) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 14:02:35.541058 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-7ad7870c-9de2-43a1-b9c5-b344fafb545e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:35.598790 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1447/2903] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jul 6 14:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 14:02:35.608839 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d15722b-93fa-40e4-bb96-7f34c178f570 admin admin] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:35.610360 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-6d15722b-93fa-40e4-bb96-7f34c178f570 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:35.628979 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:35.667295 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d15722b-93fa-40e4-bb96-7f34c178f570 admin admin] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:35.671531 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde0023-f51f-4258-8dba-4e2c96a30bc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f250f5dbc54d2e9293962ece848f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:35.672170 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde0023-f51f-4258-8dba-4e2c96a30bc8 service neutron] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:35.672454 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde0023-f51f-4258-8dba-4e2c96a30bc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:35.672731 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde0023-f51f-4258-8dba-4e2c96a30bc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:35.681754 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde0023-f51f-4258-8dba-4e2c96a30bc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:35.683166 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde0023-f51f-4258-8dba-4e2c96a30bc8 service neutron] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:35.693170 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1457/2904] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 14:02:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 14:02:35.953472 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad7870c-9de2-43a1-b9c5-b344fafb545e None None] Authenticating user token {{(pid=86850) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:02:35.955016 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.fernet_utils [None req-7ad7870c-9de2-43a1-b9c5-b344fafb545e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86850) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:02:35.970559 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad7870c-9de2-43a1-b9c5-b344fafb545e None None] Validating token access rules against request {{(pid=86850) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:02:35.975268 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0009039-8bc5-4525-9880-12fe71006a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c17e13a3d58f4ceab768f46d43e2cab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '775434dcf39c43beaa4ed6a50089f763', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86850) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:02:35.976868 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0009039-8bc5-4525-9880-12fe71006a43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:02:35.977546 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0009039-8bc5-4525-9880-12fe71006a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:02:35.977826 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0009039-8bc5-4525-9880-12fe71006a43 admin admin] PATH_INFO: `/v3/projects/a0d86d7a87e242b194b423c91adfff7c` {{(pid=86850) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:02:35.979259 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0009039-8bc5-4525-9880-12fe71006a43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0d86d7a87e242b194b423c91adfff7c)` {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:02:35.981120 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0009039-8bc5-4525-9880-12fe71006a43 admin admin] RBAC: Authorization granted {{(pid=86850) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:02:36.027812 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a0009039-8bc5-4525-9880-12fe71006a43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0d86d7a87e242b194b423c91adfff7c', 'request_id': 'req-a0009039-8bc5-4525-9880-12fe71006a43'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:36.034522 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a0009039-8bc5-4525-9880-12fe71006a43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0d86d7a87e242b194b423c91adfff7c', 'request_id': 'req-a0009039-8bc5-4525-9880-12fe71006a43'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:36.041545 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a0009039-8bc5-4525-9880-12fe71006a43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0d86d7a87e242b194b423c91adfff7c', 'request_id': 'req-a0009039-8bc5-4525-9880-12fe71006a43'} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:36.052393 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a0009039-8bc5-4525-9880-12fe71006a43 admin admin] The token cache is being invalidate because project a0d86d7a87e242b194b423c91adfff7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86850) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 14:02:36.052679 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a0009039-8bc5-4525-9880-12fe71006a43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:36.053305 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG keystone.notifications [None req-a0009039-8bc5-4525-9880-12fe71006a43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86850) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 14:02:36.055132 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: [pid: 86850|app: 0|req: 1448/2905] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jul 6 14:02:35 2022] DELETE /identity/v3/projects/a0d86d7a87e242b194b423c91adfff7c => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 14:02:45.689139 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=19 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:02:46.049408 dv-cinder-819149-8-iscsi devstack@keystone.service[86850]: DEBUG dbcounter [-] [86850] Writing DB stats keystone:SELECT=81,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86850) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 14:04:35.837914 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde0023-f51f-4258-8dba-4e2c96a30bc8 service neutron] Authenticating user token {{(pid=86849) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 14:04:35.839298 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-7bde0023-f51f-4258-8dba-4e2c96a30bc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:04:35.908447 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde0023-f51f-4258-8dba-4e2c96a30bc8 service neutron] Validating token access rules against request {{(pid=86849) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 14:04:35.966356 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa07d790-a0d4-4aac-9461-748736a17c7b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '890d2ae150d647a18dae25159cf97484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe6c3d8da2743be9609184db2506303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86849) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 14:04:35.967002 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07d790-a0d4-4aac-9461-748736a17c7b service placement] REQUEST_METHOD: `GET` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 14:04:35.967282 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07d790-a0d4-4aac-9461-748736a17c7b service placement] SCRIPT_NAME: `/identity` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 14:04:35.967555 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07d790-a0d4-4aac-9461-748736a17c7b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86849) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 14:04:35.969583 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.fernet_utils [None req-aa07d790-a0d4-4aac-9461-748736a17c7b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86849) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 14:04:35.980677 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa07d790-a0d4-4aac-9461-748736a17c7b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 14:04:35.982276 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa07d790-a0d4-4aac-9461-748736a17c7b service placement] RBAC: Authorization granted {{(pid=86849) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 14:04:35.991988 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: [pid: 86849|app: 0|req: 1458/2906] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jul 6 14:04:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 14:04:45.987740 dv-cinder-819149-8-iscsi devstack@keystone.service[86849]: DEBUG dbcounter [-] [86849] Writing DB stats keystone:SELECT=58 {{(pid=86849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}